Aug 05 17:33:11.351788 ip-10-0-131-77 systemd[1]: Starting Kubernetes Kubelet...
Aug 05 17:33:11.722944 ip-10-0-131-77 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 17:33:11.722944 ip-10-0-131-77 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Aug 05 17:33:11.722944 ip-10-0-131-77 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 17:33:11.722944 ip-10-0-131-77 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Aug 05 17:33:11.722944 ip-10-0-131-77 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 17:33:11.724708 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.724576    2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Aug 05 17:33:11.728984 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.728969    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:33:11.728984 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.728984    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.728988    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.728991    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.728994    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.728996    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.728999    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729003    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729006    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729008    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729011    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729013    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729017    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729021    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729024    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729026    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729029    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729031    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729034    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729036    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729039    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:33:11.729043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729042    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:33:11.729505 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729045    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:33:11.729505 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729047    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:33:11.729505 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729051    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:33:11.729505 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729053    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:33:11.729505 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729056    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:33:11.729505 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729059    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:33:11.729505 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729061    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:33:11.729505 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729064    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:33:11.729505 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729067    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:33:11.729505 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729069    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:33:11.729505 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729072    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:33:11.729505 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729074    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:33:11.729505 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729077    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:33:11.729505 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729079    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:33:11.729505 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729082    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:33:11.729505 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729085    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:33:11.729505 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729091    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:33:11.729505 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729093    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:33:11.729505 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729098    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729101    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729104    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729106    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729109    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729112    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729114    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729117    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729119    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729122    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729124    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729126    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729129    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729131    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729134    2569 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729137    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729139    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729142    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729144    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729147    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:33:11.729964 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729149    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729152    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729155    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729157    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729160    2569 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729162    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729165    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729167    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729170    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729172    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729174    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729177    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729179    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729182    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729185    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729187    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729189    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729192    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729194    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729197    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:33:11.730618 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729199    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:33:11.731076 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729202    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:33:11.731076 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729204    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:33:11.731076 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729206    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:33:11.731076 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729209    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:33:11.731076 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729212    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:33:11.731076 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729215    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:33:11.731076 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729218    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:33:11.731076 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729221    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:33:11.731076 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729224    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:33:11.731076 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729227    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:33:11.731076 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729230    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:33:11.731076 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729232    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:33:11.731076 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729235    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:33:11.731076 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729237    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:33:11.731076 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729240    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:33:11.731076 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729242    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:33:11.731076 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729245    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:33:11.731076 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729249    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:33:11.731076 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.729998    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730006    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730009    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730012    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730015    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730018    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730022    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730024    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730027    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730030    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730033    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730035    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730038    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730040    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730043    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730045    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730048    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730050    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730053    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730055    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:33:11.731526 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730058    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730061    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730064    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730066    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730069    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730071    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730074    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730077    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730079    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730082    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730084    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730087    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730091    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730094    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730097    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730100    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730102    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730106    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730109    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730112    2569 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:33:11.732004 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730114    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:33:11.732472 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730117    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:33:11.732472 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730119    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:33:11.732472 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730121    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:33:11.732472 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730129    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:33:11.732472 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730132    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:33:11.732472 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730135    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:33:11.732472 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730137    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:33:11.732472 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730140    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:33:11.732472 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730144    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:33:11.732472 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730147    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:33:11.732472 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730150    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:33:11.732472 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730154    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:33:11.732472 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730157    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:33:11.732472 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730159    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:33:11.732472 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730162    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:33:11.732472 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730165    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:33:11.732472 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730168    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:33:11.732472 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730170    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:33:11.732472 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730173    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:33:11.732926 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730176    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:33:11.732926 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730179    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:33:11.732926 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730182    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:33:11.732926 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730184    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:33:11.732926 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730187    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:33:11.732926 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730189    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:33:11.732926 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730192    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:33:11.732926 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730194    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:33:11.732926 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730197    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:33:11.732926 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730199    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:33:11.732926 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730201    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:33:11.732926 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730204    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:33:11.732926 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730207    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:33:11.732926 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730209    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:33:11.732926 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730212    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:33:11.732926 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730214    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:33:11.732926 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730216    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:33:11.732926 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730219    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:33:11.732926 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730221    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730224    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730226    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730229    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730232    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730234    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730238    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730240    2569 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730243    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730245    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730247    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730250    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730252    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730255    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730257    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730259    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730262    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730265    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730267    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730270    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:33:11.733373 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730272    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:33:11.733886 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730698    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:33:11.733886 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730711    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:33:11.733886 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730714    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:33:11.733886 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730716    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:33:11.733886 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730724    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:33:11.733886 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730728    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:33:11.733886 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730730    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:33:11.733886 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730733    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:33:11.733886 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730737    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:33:11.733886 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730741    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:33:11.733886 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730744    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:33:11.733886 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730747    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:33:11.733886 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730749    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:33:11.733886 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730753    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:33:11.733886 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730757    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:33:11.733886 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730760    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:33:11.733886 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730763    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:33:11.733886 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730766    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:33:11.733886 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730769    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730772    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730774    2569 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730777    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730779    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730782    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730784    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730787    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730789    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730792    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730795    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730797    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730800    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730802    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730805    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730808    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730810    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730812    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730815    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730823    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:33:11.734327 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730826    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:33:11.734828 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730828    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:33:11.734828 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730830    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:33:11.734828 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730833    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:33:11.734828 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730835    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:33:11.734828 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730837    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:33:11.734828 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730840    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:33:11.734828 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730842    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:33:11.734828 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730844    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:33:11.734828 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730847    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:33:11.734828 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730849    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:33:11.734828 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730853    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:33:11.734828 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730857    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:33:11.734828 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730859    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:33:11.734828 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730862    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:33:11.734828 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730864    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:33:11.734828 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730866    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:33:11.734828 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730869    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:33:11.734828 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730871    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:33:11.734828 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730874    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730876    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730879    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730881    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730884    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730886    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730888    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730891    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730893    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730896    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730898    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730900    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730903    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730910    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730913    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730915    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730920    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730922    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730925    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730927    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:33:11.735268 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730930    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730932    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730935    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730937    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730940    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730943    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730945    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730948    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730950    2569 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730953    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730956    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730958    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730961    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730964    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730966    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730969    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730971    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730973    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730976    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730979    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:33:11.735741 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730981    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.730984    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731053    2569 flags.go:64] FLAG: --address="0.0.0.0"
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731062    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731081    2569 flags.go:64] FLAG: --anonymous-auth="true"
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731088    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731103    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731106    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731112    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731116    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731119    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731122    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731125    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731128    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731131    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731135    2569 flags.go:64] FLAG: --cgroup-root=""
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731138    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731140    2569 flags.go:64] FLAG: --client-ca-file=""
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731143    2569 flags.go:64] FLAG: --cloud-provider="external"
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731146    2569 flags.go:64] FLAG: --cluster-dns="[]"
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731152    2569 flags.go:64] FLAG: --cluster-domain=""
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731155    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731158    2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731161    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Aug 05 17:33:11.736232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731164    2569 flags.go:64] FLAG: --container-log-max-files="5"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731167    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731170    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731173    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731176    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731179    2569 flags.go:64] FLAG: --contention-profiling="false"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731182    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731184    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731187    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731190    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731194    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731198    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731200    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731204    2569 flags.go:64] FLAG: --enable-load-reader="false"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731207    2569 flags.go:64] FLAG: --enable-server="true"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731215    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731226    2569 flags.go:64] FLAG: --event-burst="100"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731229    2569 flags.go:64] FLAG: --event-qps="50"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731232    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731235    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731237    2569 flags.go:64] FLAG: --eviction-hard=""
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731241    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731244    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731247    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731250    2569 flags.go:64] FLAG: --eviction-soft=""
Aug 05 17:33:11.736819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731253    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731255    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731258    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731261    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731264    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731266    2569 flags.go:64] FLAG: --fail-swap-on="true"
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731269    2569 flags.go:64] FLAG: --feature-gates=""
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731273    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731275    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731278    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731281    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731284    2569 flags.go:64] FLAG: --healthz-port="10248"
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731287    2569 flags.go:64] FLAG: --help="false"
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731289    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-131-77.ec2.internal"
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731292    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731295    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731298    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731301    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731304    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731307    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731310    2569 flags.go:64] FLAG: --image-service-endpoint=""
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731312    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731315    2569 flags.go:64] FLAG: --kube-api-burst="100"
Aug 05 17:33:11.737375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731324    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731328    2569 flags.go:64] FLAG: --kube-api-qps="50"
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731330    2569 flags.go:64] FLAG: --kube-reserved=""
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731333    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731336    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731339    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731343    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731346    2569 flags.go:64] FLAG: --lock-file=""
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731348    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731351    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731353    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731359    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731361    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731364    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731366    2569 flags.go:64] FLAG: --logging-format="text"
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731369    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731372    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731379    2569 flags.go:64] FLAG: --manifest-url=""
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731381    2569 flags.go:64] FLAG: --manifest-url-header=""
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731385    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731388    2569 flags.go:64] FLAG: --max-open-files="1000000"
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731396    2569 flags.go:64] FLAG: --max-pods="110"
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731399    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731402    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731404    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Aug 05 17:33:11.738098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731407    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731410    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731413    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731416    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731422    2569 flags.go:64] FLAG: --node-status-max-images="50"
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731425    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731428    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731432    2569 flags.go:64] FLAG: --pod-cidr=""
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731440    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731445    2569 flags.go:64] FLAG: --pod-manifest-path=""
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731448    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731451    2569 flags.go:64] FLAG: --pods-per-core="0"
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731454    2569 flags.go:64] FLAG: --port="10250"
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731457    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731461    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-097188584d937962e"
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731465    2569 flags.go:64] FLAG: --qos-reserved=""
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731468    2569 flags.go:64] FLAG: --read-only-port="10255"
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731471    2569 flags.go:64] FLAG: --register-node="true"
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731474    2569 flags.go:64] FLAG: --register-with-taints=""
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731477    2569 flags.go:64] FLAG: --registry-burst="10"
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731480    2569 flags.go:64] FLAG: --registry-qps="5"
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731483    2569 flags.go:64] FLAG: --reserved-cpus=""
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731485    2569 flags.go:64] FLAG: --reserved-memory=""
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731489    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731492    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Aug 05 17:33:11.739053 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731495    2569 flags.go:64] FLAG: --rotate-certificates="false"
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731497    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731500    2569 flags.go:64] FLAG: --runonce="false"
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731503    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731518    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731521    2569 flags.go:64] FLAG: --seccomp-default="false"
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731524    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731527    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731530    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731533    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731536    2569 flags.go:64] FLAG: --storage-driver-password="root"
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731539    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731542    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731544    2569 flags.go:64] FLAG: --storage-driver-user="root"
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731549    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731552    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731560    2569 flags.go:64] FLAG: --system-cgroups=""
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731563    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731568    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731571    2569 flags.go:64] FLAG: --tls-cert-file=""
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731574    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731582    2569 flags.go:64] FLAG: --tls-min-version=""
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731585    2569 flags.go:64] FLAG: --tls-private-key-file=""
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731587    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731591    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Aug 05 17:33:11.739670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731594    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731596    2569 flags.go:64] FLAG: --v="2"
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731600    2569 flags.go:64] FLAG: --version="false"
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731604    2569 flags.go:64] FLAG: --vmodule=""
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731607    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.731610    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731783    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731787    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731790    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731793    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731795    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731798    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731800    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731803    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731805    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731808    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731810    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731813    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731815    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731818    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731820    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:33:11.740281 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731830    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:33:11.740854 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731832    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:33:11.740854 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731835    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:33:11.740854 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731843    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:33:11.740854 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731846    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:33:11.740854 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731848    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:33:11.740854 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731850    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:33:11.740854 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731855    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:33:11.740854 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731858    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:33:11.740854 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731860    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:33:11.740854 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731863    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:33:11.740854 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731865    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:33:11.740854 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731868    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:33:11.740854 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731870    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:33:11.740854 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731872    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:33:11.740854 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731875    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:33:11.740854 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731877    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:33:11.740854 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731880    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:33:11.740854 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731882    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:33:11.740854 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731884    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:33:11.741289 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731888    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:33:11.741289 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731891    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:33:11.741289 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731894    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:33:11.741289 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731897    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:33:11.741289 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731900    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:33:11.741289 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731903    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:33:11.741289 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731905    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:33:11.741289 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731908    2569 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:33:11.741289 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731910    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:33:11.741289 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731913    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:33:11.741289 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731915    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:33:11.741289 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731918    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:33:11.741289 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731921    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:33:11.741289 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731924    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:33:11.741289 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731926    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:33:11.741289 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731929    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:33:11.741289 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731936    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:33:11.741289 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731939    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:33:11.741289 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731941    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:33:11.741752 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731945    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:33:11.741752 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731948    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:33:11.741752 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731950    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:33:11.741752 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731953    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:33:11.741752 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731955    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:33:11.741752 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731958    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:33:11.741752 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731960    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:33:11.741752 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731963    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:33:11.741752 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731965    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:33:11.741752 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731968    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:33:11.741752 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731970    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:33:11.741752 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731972    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:33:11.741752 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731975    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:33:11.741752 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731977    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:33:11.741752 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731980    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:33:11.741752 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731982    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:33:11.741752 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731986    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:33:11.741752 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731990    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:33:11.741752 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731992    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731995    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.731997    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732000    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732002    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732006    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732009    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732013    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732016    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732019    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732022    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732024    2569 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732032    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732035    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732039    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732041    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732044    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732046    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732049    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732051    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:33:11.742179 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732054    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:33:11.742657 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732056    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:33:11.742657 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732059    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:33:11.742657 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732061    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:33:11.742657 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732064    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:33:11.742657 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732066    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:33:11.742657 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.732069    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:33:11.742657 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.732655    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 05 17:33:11.742657 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.740680    2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Aug 05 17:33:11.742657 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.740694    2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Aug 05 17:33:11.742657 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.741431    2569 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Aug 05 17:33:11.742657 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.741439    2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Aug 05 17:33:11.742657 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741503    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:33:11.742657 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741520    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:33:11.742657 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741523    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:33:11.742985 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741526    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:33:11.742985 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741529    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:33:11.742985 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741531    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:33:11.742985 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741534    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:33:11.742985 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741537    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:33:11.742985 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741540    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:33:11.742985 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741542    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:33:11.742985 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741545    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:33:11.742985 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741550    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:33:11.742985 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741553    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:33:11.742985 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741556    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:33:11.742985 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741559    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:33:11.742985 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741563    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:33:11.742985 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741565    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:33:11.742985 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741569    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:33:11.742985 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741571    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:33:11.742985 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741574    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:33:11.742985 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741577    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:33:11.742985 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741580    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741583    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741585    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741588    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741591    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741593    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741596    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741599    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741601    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741604    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741607    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741609    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741611    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741614    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741616    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741619    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741621    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741623    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741626    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741628    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:33:11.743417 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741631    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741634    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741637    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741641    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741644    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741654    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741657    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741660    2569 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741662    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741665    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741667    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741670    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741672    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741674    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741677    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741679    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741682    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741685    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741687    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741689    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:33:11.743884 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741692    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741694    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741697    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741699    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741702    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741704    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741706    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741709    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741711    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741713    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741716    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741718    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741721    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741723    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741726    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741728    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741730    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741733    2569 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741741    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741744    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:33:11.744345 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741746    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:33:11.744868 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741749    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:33:11.744868 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741751    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:33:11.744868 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741755    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:33:11.744868 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741759    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:33:11.744868 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741762    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:33:11.744868 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741764    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:33:11.744868 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741767    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:33:11.744868 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741769    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:33:11.744868 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741772    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:33:11.744868 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741774    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:33:11.744868 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741777    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:33:11.744868 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741779    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:33:11.744868 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741781    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:33:11.744868 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741784    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:33:11.744868 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741786    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:33:11.744868 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741789    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:33:11.744868 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741791    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:33:11.745265 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.741796    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 05 17:33:11.745265 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741915    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:33:11.745265 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741920    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:33:11.745265 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741924    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:33:11.745265 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741927    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:33:11.745265 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741930    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:33:11.745265 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741932    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:33:11.745265 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741935    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:33:11.745265 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741937    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:33:11.745265 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741940    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:33:11.745265 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741943    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:33:11.745265 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741946    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:33:11.745265 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741948    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:33:11.745265 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741958    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:33:11.745265 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741961    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:33:11.745638 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741964    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:33:11.745638 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741967    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:33:11.745638 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741969    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:33:11.745638 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741971    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:33:11.745638 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741975    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:33:11.745638 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741979    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:33:11.745638 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741982    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:33:11.745638 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741985    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:33:11.745638 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741987    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:33:11.745638 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741990    2569 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:33:11.745638 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741992    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:33:11.745638 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741995    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:33:11.745638 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.741999    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:33:11.745638 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742002    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:33:11.745638 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742005    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:33:11.745638 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742007    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:33:11.745638 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742010    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:33:11.745638 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742013    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:33:11.745638 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742015    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:33:11.746063 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742018    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:33:11.746063 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742020    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:33:11.746063 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742023    2569 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:33:11.746063 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742025    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:33:11.746063 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742028    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:33:11.746063 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742031    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:33:11.746063 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742033    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:33:11.746063 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742036    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:33:11.746063 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742038    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:33:11.746063 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742041    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:33:11.746063 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742043    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:33:11.746063 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742046    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:33:11.746063 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742050    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:33:11.746063 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742052    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:33:11.746063 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742055    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:33:11.746063 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742057    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:33:11.746063 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742060    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:33:11.746063 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742062    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:33:11.746063 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742065    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742067    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742069    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742072    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742074    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742076    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742079    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742081    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742083    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742086    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742089    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742091    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742094    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742097    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742101    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742103    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742106    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742108    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742111    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742113    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:33:11.746496 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742115    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742118    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742120    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742122    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742125    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742127    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742130    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742133    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742136    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742138    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742141    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742143    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742145    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742148    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742150    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742153    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742155    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742158    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742160    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742163    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:33:11.746966 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742165    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:33:11.747418 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742167    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:33:11.747418 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742170    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:33:11.747418 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742172    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:33:11.747418 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742174    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:33:11.747418 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742177    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:33:11.747418 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742179    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:33:11.747418 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:11.742182    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:33:11.747418 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.742187    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 05 17:33:11.747418 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.742277    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Aug 05 17:33:11.747418 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.745110    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Aug 05 17:33:11.747418 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.746450    2569 server.go:1019] "Starting client certificate rotation"
Aug 05 17:33:11.747418 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.746550    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 05 17:33:11.747418 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.746578    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 05 17:33:11.748863 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.748848    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 05 17:33:11.750116 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.750105    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 05 17:33:11.759885 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.759868    2569 log.go:25] "Validated CRI v1 runtime API"
Aug 05 17:33:11.761939 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.761928    2569 log.go:25] "Validated CRI v1 image API"
Aug 05 17:33:11.763587 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.763521    2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Aug 05 17:33:11.765854 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.765835    2569 fs.go:135] Filesystem UUIDs: map[28848cb1-590a-4f56-b30f-3371defc37f1:/dev/nvme0n1p4 585af382-5782-4320-a0ad-53384c81542e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Aug 05 17:33:11.765924 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.765852    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Aug 05 17:33:11.771149 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.771040    2569 manager.go:216] Machine: {Timestamp:2026-08-05 17:33:11.769891168 +0000 UTC m=+0.306235099 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3204985 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a5f4b54a07ee0d342da8c8243da1d SystemUUID:ec2a5f4b-54a0-7ee0-d342-da8c8243da1d BootID:b447dbf1-7e59-45ac-b57d-740b4b9fb2d8 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ad:e7:10:a3:5f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ad:e7:10:a3:5f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:b6:1e:a1:75:66 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Aug 05 17:33:11.771149 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.771133    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Aug 05 17:33:11.771273 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.771200    2569 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Aug 05 17:33:11.774211 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.774184    2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Aug 05 17:33:11.774343 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.774218    2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-77.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Aug 05 17:33:11.775483 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.775470    2569 topology_manager.go:138] "Creating topology manager with none policy"
Aug 05 17:33:11.775542 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.775486    2569 container_manager_linux.go:309] "Creating device plugin manager"
Aug 05 17:33:11.775542 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.775494    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 05 17:33:11.776283 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.776273    2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 05 17:33:11.776766 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.776752    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 05 17:33:11.777393 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.777382    2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Aug 05 17:33:11.780859 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.780848    2569 state_mem.go:36] "Initialized new in-memory state store"
Aug 05 17:33:11.780964 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.780955    2569 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Aug 05 17:33:11.780994 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.780983    2569 kubelet.go:486] "Attempting to sync node with API server"
Aug 05 17:33:11.781020 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.780995    2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Aug 05 17:33:11.781020 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.781008    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Aug 05 17:33:11.781020 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.781017    2569 kubelet.go:398] "Adding apiserver pod source"
Aug 05 17:33:11.781121 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.781024    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Aug 05 17:33:11.781374 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.781362    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 05 17:33:11.781456 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.781381    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 05 17:33:11.784234 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.784219    2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Aug 05 17:33:11.785549 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.785537    2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Aug 05 17:33:11.785599 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.785566    2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Aug 05 17:33:11.786981 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.786969    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Aug 05 17:33:11.787014 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.786991    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Aug 05 17:33:11.787014 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.787001    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Aug 05 17:33:11.787014 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.787006    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Aug 05 17:33:11.787014 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.787015    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Aug 05 17:33:11.787117 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.787020    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Aug 05 17:33:11.787117 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.787028    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Aug 05 17:33:11.787117 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.787032    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Aug 05 17:33:11.787117 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.787038    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Aug 05 17:33:11.787117 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.787043    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Aug 05 17:33:11.787117 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.787051    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Aug 05 17:33:11.787117 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.787057    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Aug 05 17:33:11.787728 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.787719    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Aug 05 17:33:11.787764 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.787728    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Aug 05 17:33:11.790670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.790658    2569 server.go:1268] "Started kubelet"
Aug 05 17:33:11.790802 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.790751    2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Aug 05 17:33:11.790867 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.790826    2569 server_v1.go:49] "podresources" method="list" useActivePods=true
Aug 05 17:33:11.790867 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.790833    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Aug 05 17:33:11.791441 ip-10-0-131-77 systemd[1]: Started Kubernetes Kubelet.
Aug 05 17:33:11.792066 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.792024    2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Aug 05 17:33:11.793005 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.792990    2569 server.go:310] "Adding debug handlers to kubelet server"
Aug 05 17:33:11.794050 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.794023    2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-77.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Aug 05 17:33:11.794122 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:11.794021    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-77.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Aug 05 17:33:11.794663 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:11.794627    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Aug 05 17:33:11.796283 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:11.796266    2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Aug 05 17:33:11.798217 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.798196    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Aug 05 17:33:11.798299 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.798271    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Aug 05 17:33:11.798562 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:11.798502    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-77.ec2.internal\" not found"
Aug 05 17:33:11.799013 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.798992    2569 volume_manager.go:311] "The desired_state_of_world populator starts"
Aug 05 17:33:11.799082 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.799016    2569 volume_manager.go:313] "Starting Kubelet Volume Manager"
Aug 05 17:33:11.799203 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.799188    2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Aug 05 17:33:11.799299 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.799243    2569 reconstruct.go:97] "Volume reconstruction finished"
Aug 05 17:33:11.799299 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.799256    2569 reconciler.go:29] "Reconciler: start to sync state"
Aug 05 17:33:11.804052 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.804030    2569 factory.go:153] Registering CRI-O factory
Aug 05 17:33:11.804052 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.804055    2569 factory.go:223] Registration of the crio container factory successfully
Aug 05 17:33:11.804192 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:11.800291    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-77.ec2.internal.18c8f92584ddbf32  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-77.ec2.internal,UID:ip-10-0-131-77.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-77.ec2.internal,},FirstTimestamp:2026-08-05 17:33:11.790632754 +0000 UTC m=+0.326976684,LastTimestamp:2026-08-05 17:33:11.790632754 +0000 UTC m=+0.326976684,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-77.ec2.internal,}"
Aug 05 17:33:11.804192 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.804107    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jk8lw"
Aug 05 17:33:11.804192 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.804125    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Aug 05 17:33:11.804192 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.804135    2569 factory.go:55] Registering systemd factory
Aug 05 17:33:11.804192 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.804142    2569 factory.go:223] Registration of the systemd container factory successfully
Aug 05 17:33:11.804192 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.804160    2569 factory.go:103] Registering Raw factory
Aug 05 17:33:11.804192 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.804173    2569 manager.go:1196] Started watching for new ooms in manager
Aug 05 17:33:11.804767 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.804752    2569 manager.go:349] Starting recovery of all containers
Aug 05 17:33:11.806709 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:11.806681    2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-77.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Aug 05 17:33:11.807023 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:11.806994    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Aug 05 17:33:11.813767 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.813747    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jk8lw"
Aug 05 17:33:11.814983 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.814969    2569 manager.go:354] Recovery completed
Aug 05 17:33:11.819157 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.819141    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:33:11.821000 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.820898    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:33:11.821069 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.821014    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:33:11.821069 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.821024    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:33:11.821551 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.821535    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Aug 05 17:33:11.821551 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.821550    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Aug 05 17:33:11.821667 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.821572    2569 state_mem.go:36] "Initialized new in-memory state store"
Aug 05 17:33:11.821667 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.821543    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:33:11.822394 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.822380    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:33:11.822478 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.822403    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:33:11.822478 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.822411    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:33:11.822652 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.822640    2569 policy_none.go:49] "None policy: Start"
Aug 05 17:33:11.822709 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.822655    2569 memory_manager.go:187] "Starting memorymanager" policy="None"
Aug 05 17:33:11.822709 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.822664    2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Aug 05 17:33:11.823601 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.823586    2569 policy_none.go:47] "Start"
Aug 05 17:33:11.823697 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:11.823567    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-77.ec2.internal.18c8f92586ad1b59  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-77.ec2.internal,UID:ip-10-0-131-77.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-77.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-77.ec2.internal,},FirstTimestamp:2026-08-05 17:33:11.820999513 +0000 UTC m=+0.357343447,LastTimestamp:2026-08-05 17:33:11.820999513 +0000 UTC m=+0.357343447,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-77.ec2.internal,}"
Aug 05 17:33:11.861037 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.861022    2569 manager.go:337] "Starting Device Plugin manager"
Aug 05 17:33:11.861109 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:11.861050    2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Aug 05 17:33:11.861109 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.861059    2569 server.go:87] "Starting device plugin registration server"
Aug 05 17:33:11.861378 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.861364    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Aug 05 17:33:11.861461 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.861379    2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Aug 05 17:33:11.861527 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.861460    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Aug 05 17:33:11.861576 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.861542    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Aug 05 17:33:11.861576 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.861552    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Aug 05 17:33:11.862062 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:11.862040    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Aug 05 17:33:11.862135 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:11.862098    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-77.ec2.internal\" not found"
Aug 05 17:33:11.959097 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.959069    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Aug 05 17:33:11.960281 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.960264    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Aug 05 17:33:11.960334 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.960311    2569 status_manager.go:244] "Starting to sync pod status with apiserver"
Aug 05 17:33:11.960334 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.960333    2569 kubelet.go:2448] "Starting kubelet main sync loop"
Aug 05 17:33:11.960392 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:11.960371    2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Aug 05 17:33:11.961693 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.961679    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:33:11.962552 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.962537    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:33:11.962689 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.962567    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:33:11.962689 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.962583    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:33:11.962689 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.962610    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-77.ec2.internal"
Aug 05 17:33:11.963140 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.963125    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:33:11.976569 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:11.976524    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-77.ec2.internal"
Aug 05 17:33:11.976569 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:11.976549    2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-77.ec2.internal\": node \"ip-10-0-131-77.ec2.internal\" not found"
Aug 05 17:33:11.999746 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:11.999731    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-77.ec2.internal\" not found"
Aug 05 17:33:12.061066 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.060405    2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-77.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal"]
Aug 05 17:33:12.061176 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.061110    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:33:12.061954 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.061935    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:33:12.062074 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.061965    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:33:12.062074 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.061977    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:33:12.063207 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.063193    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:33:12.063306 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.063292    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-77.ec2.internal"
Aug 05 17:33:12.063306 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.063318    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:33:12.063865 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.063851    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:33:12.063954 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.063855    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:33:12.063954 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.063893    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:33:12.063954 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.063906    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:33:12.063954 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.063874    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:33:12.064120 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.063960    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:33:12.065297 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.065281    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal"
Aug 05 17:33:12.065374 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.065312    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:33:12.065928 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.065914    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:33:12.066019 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.065932    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:33:12.066019 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.065942    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:33:12.092116 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:12.092093    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-77.ec2.internal\" not found" node="ip-10-0-131-77.ec2.internal"
Aug 05 17:33:12.096189 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:12.096172    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-77.ec2.internal\" not found" node="ip-10-0-131-77.ec2.internal"
Aug 05 17:33:12.099791 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:12.099776    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-77.ec2.internal\" not found"
Aug 05 17:33:12.199827 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:12.199805    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-77.ec2.internal\" not found"
Aug 05 17:33:12.199827 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.199817    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b12f1453a65ccfd781f199e0071a899-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal\" (UID: \"4b12f1453a65ccfd781f199e0071a899\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal"
Aug 05 17:33:12.199951 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.199841    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/beae357a1d1b622108905e09a0f5f3aa-config\") pod \"kube-apiserver-proxy-ip-10-0-131-77.ec2.internal\" (UID: \"beae357a1d1b622108905e09a0f5f3aa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-77.ec2.internal"
Aug 05 17:33:12.199951 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.199856    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4b12f1453a65ccfd781f199e0071a899-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal\" (UID: \"4b12f1453a65ccfd781f199e0071a899\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal"
Aug 05 17:33:12.300678 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:12.300625    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-77.ec2.internal\" not found"
Aug 05 17:33:12.300678 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.300658    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/beae357a1d1b622108905e09a0f5f3aa-config\") pod \"kube-apiserver-proxy-ip-10-0-131-77.ec2.internal\" (UID: \"beae357a1d1b622108905e09a0f5f3aa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-77.ec2.internal"
Aug 05 17:33:12.300678 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.300678    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4b12f1453a65ccfd781f199e0071a899-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal\" (UID: \"4b12f1453a65ccfd781f199e0071a899\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal"
Aug 05 17:33:12.300841 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.300694    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b12f1453a65ccfd781f199e0071a899-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal\" (UID: \"4b12f1453a65ccfd781f199e0071a899\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal"
Aug 05 17:33:12.300841 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.300727    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b12f1453a65ccfd781f199e0071a899-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal\" (UID: \"4b12f1453a65ccfd781f199e0071a899\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal"
Aug 05 17:33:12.300841 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.300761    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/beae357a1d1b622108905e09a0f5f3aa-config\") pod \"kube-apiserver-proxy-ip-10-0-131-77.ec2.internal\" (UID: \"beae357a1d1b622108905e09a0f5f3aa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-77.ec2.internal"
Aug 05 17:33:12.300841 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.300773    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4b12f1453a65ccfd781f199e0071a899-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal\" (UID: \"4b12f1453a65ccfd781f199e0071a899\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal"
Aug 05 17:33:12.394865 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.394838    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-77.ec2.internal"
Aug 05 17:33:12.399075 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.398933    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal"
Aug 05 17:33:12.401098 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:12.401082    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-77.ec2.internal\" not found"
Aug 05 17:33:12.501577 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:12.501555    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-77.ec2.internal\" not found"
Aug 05 17:33:12.601984 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:12.601943    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-77.ec2.internal\" not found"
Aug 05 17:33:12.702316 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:12.702297    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-77.ec2.internal\" not found"
Aug 05 17:33:12.745396 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.745374    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Aug 05 17:33:12.745776 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.745541    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 05 17:33:12.803400 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:12.803379    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-77.ec2.internal\" not found"
Aug 05 17:33:12.819061 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.819031    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-04 17:28:12 +0000 UTC" deadline="2028-05-20 08:18:23.677008968 +0000 UTC"
Aug 05 17:33:12.819061 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.819057    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15686h45m10.8579564s"
Aug 05 17:33:12.864151 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.864096    2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:33:12.900172 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.900087    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-77.ec2.internal"
Aug 05 17:33:12.908251 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.908236    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 05 17:33:12.909812 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.909800    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal"
Aug 05 17:33:12.921033 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.921017    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 05 17:33:12.942092 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.942076    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:33:12.975462 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:12.975438    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b12f1453a65ccfd781f199e0071a899.slice/crio-22e19bbdfbdb08a7d91a6385db2bba69b44b1351093c3ffd93130ab4d952ac15 WatchSource:0}: Error finding container 22e19bbdfbdb08a7d91a6385db2bba69b44b1351093c3ffd93130ab4d952ac15: Status 404 returned error can't find the container with id 22e19bbdfbdb08a7d91a6385db2bba69b44b1351093c3ffd93130ab4d952ac15
Aug 05 17:33:12.975771 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:12.975756    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeae357a1d1b622108905e09a0f5f3aa.slice/crio-2af9798aa79b6172f38343d7072d2179f4ad8b4329349bf0b7d12524ed515b7f WatchSource:0}: Error finding container 2af9798aa79b6172f38343d7072d2179f4ad8b4329349bf0b7d12524ed515b7f: Status 404 returned error can't find the container with id 2af9798aa79b6172f38343d7072d2179f4ad8b4329349bf0b7d12524ed515b7f
Aug 05 17:33:12.979232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:12.979219    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:33:14.572426 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.572403    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:33:14.965751 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.965469    2569 apiserver.go:52] "Watching apiserver"
Aug 05 17:33:14.974374 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.974352    2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Aug 05 17:33:14.974808 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.974782    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-42zkd","openshift-multus/network-metrics-daemon-x7hnn","openshift-ovn-kubernetes/ovnkube-node-xpsg4","kube-system/konnectivity-agent-4tkxv","kube-system/kube-apiserver-proxy-ip-10-0-131-77.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx","openshift-cluster-node-tuning-operator/tuned-gfl7r","openshift-image-registry/node-ca-4jm79","openshift-multus/multus-vtq9n","openshift-network-diagnostics/network-check-target-9vkxl","openshift-network-operator/iptables-alerter-dp52t","kube-system/global-pull-secret-syncer-2z2wd","openshift-dns/node-resolver-v8pbg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal"]
Aug 05 17:33:14.976480 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.976463    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:14.978601 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.978584    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dp52t"
Aug 05 17:33:14.979514 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.979497    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Aug 05 17:33:14.979756 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.979741    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2s8jt\""
Aug 05 17:33:14.979997 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.979956    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Aug 05 17:33:14.984349 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.983273    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Aug 05 17:33:14.985236 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.984788    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Aug 05 17:33:14.985325 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.985296    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mwnf7\""
Aug 05 17:33:14.985708 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.985471    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Aug 05 17:33:14.985708 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.985671    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.985848 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.985712    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Aug 05 17:33:14.988315 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.988298    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4tkxv"
Aug 05 17:33:14.989121 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.989106    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4jm79"
Aug 05 17:33:14.990455 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.990437    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:14.991593 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991574    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-run\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:14.991681 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991599    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-tmp\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:14.991681 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991616    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-slash\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.991681 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991631    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-run-ovn\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.991681 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991646    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.991681 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991662    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-sysctl-conf\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:14.991681 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991675    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-systemd\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:14.992031 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991689    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-ovnkube-script-lib\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.992031 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991707    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-modprobe-d\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:14.992031 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991720    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-sys\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:14.992031 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991733    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-host\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:14.992031 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991746    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-tuned\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:14.992031 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991760    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-ovn-node-metrics-cert\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.992031 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991810    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv46v\" (UniqueName: \"kubernetes.io/projected/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-kube-api-access-wv46v\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.992031 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991824    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-sysconfig\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:14.992031 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991838    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fc34ee1d-5676-425e-b5f7-793ec5b14b64-iptables-alerter-script\") pod \"iptables-alerter-dp52t\" (UID: \"fc34ee1d-5676-425e-b5f7-793ec5b14b64\") " pod="openshift-network-operator/iptables-alerter-dp52t"
Aug 05 17:33:14.992031 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991852    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a043e58e-b56d-4a56-b92b-7fb8f70694bc-agent-certs\") pod \"konnectivity-agent-4tkxv\" (UID: \"a043e58e-b56d-4a56-b92b-7fb8f70694bc\") " pod="kube-system/konnectivity-agent-4tkxv"
Aug 05 17:33:14.992031 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991871    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-sysctl-d\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:14.992031 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991886    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-var-lib-kubelet\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:14.992031 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991899    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc34ee1d-5676-425e-b5f7-793ec5b14b64-host-slash\") pod \"iptables-alerter-dp52t\" (UID: \"fc34ee1d-5676-425e-b5f7-793ec5b14b64\") " pod="openshift-network-operator/iptables-alerter-dp52t"
Aug 05 17:33:14.992031 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991912    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-etc-openvswitch\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.992031 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991925    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-run-openvswitch\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.992031 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991937    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-log-socket\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.992999 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991950    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-cni-bin\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.992999 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991963    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-env-overrides\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.992999 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991975    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-lib-modules\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:14.992999 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.991988    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rhkh\" (UniqueName: \"kubernetes.io/projected/fc34ee1d-5676-425e-b5f7-793ec5b14b64-kube-api-access-4rhkh\") pod \"iptables-alerter-dp52t\" (UID: \"fc34ee1d-5676-425e-b5f7-793ec5b14b64\") " pod="openshift-network-operator/iptables-alerter-dp52t"
Aug 05 17:33:14.992999 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.992001    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-var-lib-openvswitch\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.992999 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.992020    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-cni-netd\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.992999 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.992044    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-ovnkube-config\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.992999 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.992060    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a043e58e-b56d-4a56-b92b-7fb8f70694bc-konnectivity-ca\") pod \"konnectivity-agent-4tkxv\" (UID: \"a043e58e-b56d-4a56-b92b-7fb8f70694bc\") " pod="kube-system/konnectivity-agent-4tkxv"
Aug 05 17:33:14.992999 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.992073    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-kubernetes\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:14.992999 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.992095    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zljl\" (UniqueName: \"kubernetes.io/projected/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-kube-api-access-7zljl\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:14.992999 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.992111    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-kubelet\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.992999 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.992135    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-systemd-units\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.992999 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.992149    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-run-systemd\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.992999 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.992163    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-run-netns\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.992999 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.992198    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-node-log\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.992999 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.992222    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:14.992999 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.992423    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:14.993792 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.993229    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Aug 05 17:33:14.993792 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.993497    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-v8pbg"
Aug 05 17:33:14.993792 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.993702    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Aug 05 17:33:14.993929 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.993915    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Aug 05 17:33:14.994623 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.994605    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Aug 05 17:33:14.994925 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.994911    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:14.995009 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:14.994980    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:14.996506 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.996373    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:14.996506 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:14.996415    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.996911    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.997128    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.997351    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.997546    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.997721    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.997899    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.998957    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.999154    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zb8qq\""
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.999339    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.999595    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rkks2\""
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.999844    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:14.999939    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.000219    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wmz8g\""
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.000243    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.000386    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.000391    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.000399    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.000450    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:15.000447    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.000221    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rfpd7\""
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.000577    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cp654\""
Aug 05 17:33:15.000661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.000596    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Aug 05 17:33:15.001444 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.000724    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Aug 05 17:33:15.001444 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.001031    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Aug 05 17:33:15.001444 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.001155    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kcnzs\""
Aug 05 17:33:15.003758 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.003449    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Aug 05 17:33:15.004991 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.004487    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Aug 05 17:33:15.004991 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.004752    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-27cxz\""
Aug 05 17:33:15.011857 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.011631    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 05 17:33:15.027286 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.027267    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-s26tg"
Aug 05 17:33:15.041308 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.041290    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-s26tg"
Aug 05 17:33:15.084085 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.084044    2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Aug 05 17:33:15.092963 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.092939    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.093083 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.092973    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j5n6\" (UniqueName: \"kubernetes.io/projected/edf39462-7bb0-4c83-8979-c71a9df1caba-kube-api-access-9j5n6\") pod \"node-resolver-v8pbg\" (UID: \"edf39462-7bb0-4c83-8979-c71a9df1caba\") " pod="openshift-dns/node-resolver-v8pbg"
Aug 05 17:33:15.093083 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.092997    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-sys-fs\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.093083 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093023    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-cnibin\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.093083 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093062    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.093304 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093071    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-host-var-lib-cni-bin\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.093304 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093113    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-os-release\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.093304 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093147    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-run\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.093304 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093194    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-run\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.093304 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093230    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-tmp\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.093304 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093272    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bc73ab72-c095-4e0d-bbd9-b5296cadc187-dbus\") pod \"global-pull-secret-syncer-2z2wd\" (UID: \"bc73ab72-c095-4e0d-bbd9-b5296cadc187\") " pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:15.093570 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093373    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06d69006-abdd-452a-8aba-5f50ec48a6c1-multus-daemon-config\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.093570 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093398    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq2gt\" (UniqueName: \"kubernetes.io/projected/fab79c82-8579-4ce6-9410-89fce50e5b70-kube-api-access-dq2gt\") pod \"network-metrics-daemon-x7hnn\" (UID: \"fab79c82-8579-4ce6-9410-89fce50e5b70\") " pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:15.093570 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093435    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.093570 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093466    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-sysctl-conf\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.093570 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093507    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-systemd\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.093570 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093532    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-ovnkube-script-lib\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.093570 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093532    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Aug 05 17:33:15.093890 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093582    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-etc-selinux\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.093890 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093603    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.093890 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093623    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-modprobe-d\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.093890 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093634    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-sysctl-conf\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.093890 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093642    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-socket-dir\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.093890 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093752    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-modprobe-d\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.093890 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093806    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-systemd\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.093890 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093846    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-system-cni-dir\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.093890 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093882    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-sysconfig\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.094361 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093910    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-multus-conf-dir\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.094361 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093962    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-sysconfig\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.094361 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.093982    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-system-cni-dir\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.094361 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094015    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13b7a2b4-3185-4eae-98d8-855f2325e268-host\") pod \"node-ca-4jm79\" (UID: \"13b7a2b4-3185-4eae-98d8-855f2325e268\") " pod="openshift-image-registry/node-ca-4jm79"
Aug 05 17:33:15.094361 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094040    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-sysctl-d\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.094361 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094066    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc34ee1d-5676-425e-b5f7-793ec5b14b64-host-slash\") pod \"iptables-alerter-dp52t\" (UID: \"fc34ee1d-5676-425e-b5f7-793ec5b14b64\") " pod="openshift-network-operator/iptables-alerter-dp52t"
Aug 05 17:33:15.094361 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094095    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-etc-openvswitch\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.094361 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094116    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-run-openvswitch\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.094361 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094138    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-cni-bin\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.094361 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094139    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc34ee1d-5676-425e-b5f7-793ec5b14b64-host-slash\") pod \"iptables-alerter-dp52t\" (UID: \"fc34ee1d-5676-425e-b5f7-793ec5b14b64\") " pod="openshift-network-operator/iptables-alerter-dp52t"
Aug 05 17:33:15.094361 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094162    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-host-run-k8s-cni-cncf-io\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.094361 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094220    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-hostroot\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.094361 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094240    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbkhz\" (UniqueName: \"kubernetes.io/projected/13b7a2b4-3185-4eae-98d8-855f2325e268-kube-api-access-cbkhz\") pod \"node-ca-4jm79\" (UID: \"13b7a2b4-3185-4eae-98d8-855f2325e268\") " pod="openshift-image-registry/node-ca-4jm79"
Aug 05 17:33:15.094361 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094263    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-lib-modules\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.094361 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094263    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-ovnkube-script-lib\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.094361 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094284    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-kubernetes\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.094361 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094268    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-sysctl-d\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.095064 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094309    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-run-openvswitch\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.095064 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094317    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-systemd-units\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.095064 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094345    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-etc-openvswitch\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.095064 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094350    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-run-systemd\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.095064 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094360    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-systemd-units\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.095064 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094378    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-host-run-netns\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.095064 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094398    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-kubernetes\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.095064 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094401    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-host-var-lib-kubelet\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.095064 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094430    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/edf39462-7bb0-4c83-8979-c71a9df1caba-hosts-file\") pod \"node-resolver-v8pbg\" (UID: \"edf39462-7bb0-4c83-8979-c71a9df1caba\") " pod="openshift-dns/node-resolver-v8pbg"
Aug 05 17:33:15.095064 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094442    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-lib-modules\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.095064 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094446    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-cni-bin\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.095064 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094466    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-run-systemd\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.095064 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094478    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.095064 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094511    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-device-dir\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.095064 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094540    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjmzx\" (UniqueName: \"kubernetes.io/projected/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-kube-api-access-hjmzx\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.095064 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094574    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-slash\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.095064 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094614    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-run-ovn\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.095883 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094647    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-slash\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.095883 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094698    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.095883 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094728    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.095883 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094772    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-etc-kubernetes\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.095883 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094801    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-run-ovn\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.095883 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094833    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-922kd\" (UniqueName: \"kubernetes.io/projected/06d69006-abdd-452a-8aba-5f50ec48a6c1-kube-api-access-922kd\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.095883 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094861    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-registration-dir\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.095883 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094876    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-cnibin\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.095883 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094889    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.095883 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094906    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-sys\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.095883 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094942    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-host\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.095883 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094952    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-sys\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.095883 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094981    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-tuned\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.095883 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.094998    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-host\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.095883 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095031    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-ovn-node-metrics-cert\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.095883 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095076    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv46v\" (UniqueName: \"kubernetes.io/projected/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-kube-api-access-wv46v\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.095883 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095199    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/edf39462-7bb0-4c83-8979-c71a9df1caba-tmp-dir\") pod \"node-resolver-v8pbg\" (UID: \"edf39462-7bb0-4c83-8979-c71a9df1caba\") " pod="openshift-dns/node-resolver-v8pbg"
Aug 05 17:33:15.096631 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095227    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret\") pod \"global-pull-secret-syncer-2z2wd\" (UID: \"bc73ab72-c095-4e0d-bbd9-b5296cadc187\") " pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:15.096631 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095263    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fc34ee1d-5676-425e-b5f7-793ec5b14b64-iptables-alerter-script\") pod \"iptables-alerter-dp52t\" (UID: \"fc34ee1d-5676-425e-b5f7-793ec5b14b64\") " pod="openshift-network-operator/iptables-alerter-dp52t"
Aug 05 17:33:15.096631 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095288    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a043e58e-b56d-4a56-b92b-7fb8f70694bc-agent-certs\") pod \"konnectivity-agent-4tkxv\" (UID: \"a043e58e-b56d-4a56-b92b-7fb8f70694bc\") " pod="kube-system/konnectivity-agent-4tkxv"
Aug 05 17:33:15.096631 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095312    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5qpl\" (UniqueName: \"kubernetes.io/projected/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-kube-api-access-n5qpl\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.096631 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095333    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-os-release\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.096631 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095375    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06d69006-abdd-452a-8aba-5f50ec48a6c1-cni-binary-copy\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.096631 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095398    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-host-var-lib-cni-multus\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.096631 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095420    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.096631 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095444    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-var-lib-kubelet\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.096631 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095466    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-log-socket\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.096631 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095489    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-env-overrides\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.096631 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095511    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a043e58e-b56d-4a56-b92b-7fb8f70694bc-konnectivity-ca\") pod \"konnectivity-agent-4tkxv\" (UID: \"a043e58e-b56d-4a56-b92b-7fb8f70694bc\") " pod="kube-system/konnectivity-agent-4tkxv"
Aug 05 17:33:15.096631 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095533    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-multus-socket-dir-parent\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.096631 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095583    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqqc4\" (UniqueName: \"kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4\") pod \"network-check-target-9vkxl\" (UID: \"e538a505-e584-496e-9907-e2fb84f0e872\") " pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:15.096631 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095607    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rhkh\" (UniqueName: \"kubernetes.io/projected/fc34ee1d-5676-425e-b5f7-793ec5b14b64-kube-api-access-4rhkh\") pod \"iptables-alerter-dp52t\" (UID: \"fc34ee1d-5676-425e-b5f7-793ec5b14b64\") " pod="openshift-network-operator/iptables-alerter-dp52t"
Aug 05 17:33:15.096631 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095630    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-var-lib-openvswitch\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.096631 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095653    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-cni-netd\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.097490 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095674    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-ovnkube-config\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.097490 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095696    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bc73ab72-c095-4e0d-bbd9-b5296cadc187-kubelet-config\") pod \"global-pull-secret-syncer-2z2wd\" (UID: \"bc73ab72-c095-4e0d-bbd9-b5296cadc187\") " pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:15.097490 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095720    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-multus-cni-dir\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.097490 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095743    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-host-run-multus-certs\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.097490 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095766    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zljl\" (UniqueName: \"kubernetes.io/projected/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-kube-api-access-7zljl\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.097490 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095791    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-kubelet\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.097490 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095808    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs\") pod \"network-metrics-daemon-x7hnn\" (UID: \"fab79c82-8579-4ce6-9410-89fce50e5b70\") " pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:15.097490 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095848    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/13b7a2b4-3185-4eae-98d8-855f2325e268-serviceca\") pod \"node-ca-4jm79\" (UID: \"13b7a2b4-3185-4eae-98d8-855f2325e268\") " pod="openshift-image-registry/node-ca-4jm79"
Aug 05 17:33:15.097490 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095867    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-run-netns\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.097490 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095881    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-node-log\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.097490 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095931    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-node-log\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.097490 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095971    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-var-lib-kubelet\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.097490 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.095991    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-log-socket\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.097490 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.096277    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-env-overrides\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.097490 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.096638    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-cni-netd\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.097490 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.096669    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-kubelet\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.097490 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.096716    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-host-run-netns\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.098113 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.096727    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fc34ee1d-5676-425e-b5f7-793ec5b14b64-iptables-alerter-script\") pod \"iptables-alerter-dp52t\" (UID: \"fc34ee1d-5676-425e-b5f7-793ec5b14b64\") " pod="openshift-network-operator/iptables-alerter-dp52t"
Aug 05 17:33:15.098113 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.096754    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a043e58e-b56d-4a56-b92b-7fb8f70694bc-konnectivity-ca\") pod \"konnectivity-agent-4tkxv\" (UID: \"a043e58e-b56d-4a56-b92b-7fb8f70694bc\") " pod="kube-system/konnectivity-agent-4tkxv"
Aug 05 17:33:15.098113 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.096961    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-tmp\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.098113 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.097025    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-var-lib-openvswitch\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.098113 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.097219    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-ovnkube-config\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.100193 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.100156    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-ovn-node-metrics-cert\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.100285 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.100235    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-etc-tuned\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.100887 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.100849    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a043e58e-b56d-4a56-b92b-7fb8f70694bc-agent-certs\") pod \"konnectivity-agent-4tkxv\" (UID: \"a043e58e-b56d-4a56-b92b-7fb8f70694bc\") " pod="kube-system/konnectivity-agent-4tkxv"
Aug 05 17:33:15.109547 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.109494    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rhkh\" (UniqueName: \"kubernetes.io/projected/fc34ee1d-5676-425e-b5f7-793ec5b14b64-kube-api-access-4rhkh\") pod \"iptables-alerter-dp52t\" (UID: \"fc34ee1d-5676-425e-b5f7-793ec5b14b64\") " pod="openshift-network-operator/iptables-alerter-dp52t"
Aug 05 17:33:15.109547 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.109535    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zljl\" (UniqueName: \"kubernetes.io/projected/f6826678-5aa6-47c3-8cf8-ed7eb86a51c2-kube-api-access-7zljl\") pod \"tuned-gfl7r\" (UID: \"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2\") " pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.110852 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.110829    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv46v\" (UniqueName: \"kubernetes.io/projected/5b61ea1d-7181-4a3c-8bd3-989ce74d20a1-kube-api-access-wv46v\") pod \"ovnkube-node-xpsg4\" (UID: \"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.149664 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.149619    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-77.ec2.internal" event={"ID":"beae357a1d1b622108905e09a0f5f3aa","Type":"ContainerStarted","Data":"2af9798aa79b6172f38343d7072d2179f4ad8b4329349bf0b7d12524ed515b7f"}
Aug 05 17:33:15.151877 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.151855    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal" event={"ID":"4b12f1453a65ccfd781f199e0071a899","Type":"ContainerStarted","Data":"22e19bbdfbdb08a7d91a6385db2bba69b44b1351093c3ffd93130ab4d952ac15"}
Aug 05 17:33:15.196296 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196273    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-socket-dir\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.196398 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196306    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-system-cni-dir\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.196398 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196330    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-multus-conf-dir\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.196398 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196351    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-system-cni-dir\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.196398 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196374    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13b7a2b4-3185-4eae-98d8-855f2325e268-host\") pod \"node-ca-4jm79\" (UID: \"13b7a2b4-3185-4eae-98d8-855f2325e268\") " pod="openshift-image-registry/node-ca-4jm79"
Aug 05 17:33:15.196633 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196406    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-host-run-k8s-cni-cncf-io\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.196633 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196426    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-hostroot\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.196633 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196448    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbkhz\" (UniqueName: \"kubernetes.io/projected/13b7a2b4-3185-4eae-98d8-855f2325e268-kube-api-access-cbkhz\") pod \"node-ca-4jm79\" (UID: \"13b7a2b4-3185-4eae-98d8-855f2325e268\") " pod="openshift-image-registry/node-ca-4jm79"
Aug 05 17:33:15.196633 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196472    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-host-run-netns\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.196633 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196496    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-host-var-lib-kubelet\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.196633 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196521    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/edf39462-7bb0-4c83-8979-c71a9df1caba-hosts-file\") pod \"node-resolver-v8pbg\" (UID: \"edf39462-7bb0-4c83-8979-c71a9df1caba\") " pod="openshift-dns/node-resolver-v8pbg"
Aug 05 17:33:15.196633 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196544    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.196633 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196567    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-device-dir\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.196633 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196591    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjmzx\" (UniqueName: \"kubernetes.io/projected/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-kube-api-access-hjmzx\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.196633 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196619    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-etc-kubernetes\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.197242 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196641    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-922kd\" (UniqueName: \"kubernetes.io/projected/06d69006-abdd-452a-8aba-5f50ec48a6c1-kube-api-access-922kd\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.197242 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196664    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-registration-dir\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.197242 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196710    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-cnibin\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.197242 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196736    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.197242 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196767    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/edf39462-7bb0-4c83-8979-c71a9df1caba-tmp-dir\") pod \"node-resolver-v8pbg\" (UID: \"edf39462-7bb0-4c83-8979-c71a9df1caba\") " pod="openshift-dns/node-resolver-v8pbg"
Aug 05 17:33:15.197242 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196792    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret\") pod \"global-pull-secret-syncer-2z2wd\" (UID: \"bc73ab72-c095-4e0d-bbd9-b5296cadc187\") " pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:15.197242 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196818    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5qpl\" (UniqueName: \"kubernetes.io/projected/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-kube-api-access-n5qpl\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.197242 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196843    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-os-release\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.197242 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196866    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06d69006-abdd-452a-8aba-5f50ec48a6c1-cni-binary-copy\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.197242 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196889    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-host-var-lib-cni-multus\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.197242 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196916    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.197242 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196945    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-multus-socket-dir-parent\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.197242 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.196970    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqqc4\" (UniqueName: \"kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4\") pod \"network-check-target-9vkxl\" (UID: \"e538a505-e584-496e-9907-e2fb84f0e872\") " pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:15.197242 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197001    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bc73ab72-c095-4e0d-bbd9-b5296cadc187-kubelet-config\") pod \"global-pull-secret-syncer-2z2wd\" (UID: \"bc73ab72-c095-4e0d-bbd9-b5296cadc187\") " pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:15.197242 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197025    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-multus-cni-dir\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.197242 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197048    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-host-run-multus-certs\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.197242 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197085    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs\") pod \"network-metrics-daemon-x7hnn\" (UID: \"fab79c82-8579-4ce6-9410-89fce50e5b70\") " pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:15.198157 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197108    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/13b7a2b4-3185-4eae-98d8-855f2325e268-serviceca\") pod \"node-ca-4jm79\" (UID: \"13b7a2b4-3185-4eae-98d8-855f2325e268\") " pod="openshift-image-registry/node-ca-4jm79"
Aug 05 17:33:15.198157 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197135    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j5n6\" (UniqueName: \"kubernetes.io/projected/edf39462-7bb0-4c83-8979-c71a9df1caba-kube-api-access-9j5n6\") pod \"node-resolver-v8pbg\" (UID: \"edf39462-7bb0-4c83-8979-c71a9df1caba\") " pod="openshift-dns/node-resolver-v8pbg"
Aug 05 17:33:15.198157 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197159    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-sys-fs\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.198157 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197294    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-host-run-netns\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.198157 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197352    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-registration-dir\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.198157 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197360    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/edf39462-7bb0-4c83-8979-c71a9df1caba-hosts-file\") pod \"node-resolver-v8pbg\" (UID: \"edf39462-7bb0-4c83-8979-c71a9df1caba\") " pod="openshift-dns/node-resolver-v8pbg"
Aug 05 17:33:15.198157 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197405    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-host-var-lib-kubelet\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.198157 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197408    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-multus-cni-dir\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.198157 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197465    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bc73ab72-c095-4e0d-bbd9-b5296cadc187-kubelet-config\") pod \"global-pull-secret-syncer-2z2wd\" (UID: \"bc73ab72-c095-4e0d-bbd9-b5296cadc187\") " pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:15.198157 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197504    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-etc-kubernetes\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.198157 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197513    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-socket-dir\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.198157 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:15.197553    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:33:15.198157 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197561    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-system-cni-dir\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.198157 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197595    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/edf39462-7bb0-4c83-8979-c71a9df1caba-tmp-dir\") pod \"node-resolver-v8pbg\" (UID: \"edf39462-7bb0-4c83-8979-c71a9df1caba\") " pod="openshift-dns/node-resolver-v8pbg"
Aug 05 17:33:15.198157 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197600    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-multus-conf-dir\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.198157 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:15.197644    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs podName:fab79c82-8579-4ce6-9410-89fce50e5b70 nodeName:}" failed. No retries permitted until 2026-08-05 17:33:15.697597996 +0000 UTC m=+3.050272573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs") pod "network-metrics-daemon-x7hnn" (UID: "fab79c82-8579-4ce6-9410-89fce50e5b70") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:33:15.198157 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197673    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13b7a2b4-3185-4eae-98d8-855f2325e268-host\") pod \"node-ca-4jm79\" (UID: \"13b7a2b4-3185-4eae-98d8-855f2325e268\") " pod="openshift-image-registry/node-ca-4jm79"
Aug 05 17:33:15.198157 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197685    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-host-run-multus-certs\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.199017 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:15.197709    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:33:15.199017 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197741    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-hostroot\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.199017 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:15.197750    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret podName:bc73ab72-c095-4e0d-bbd9-b5296cadc187 nodeName:}" failed. No retries permitted until 2026-08-05 17:33:15.69773685 +0000 UTC m=+3.050411425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret") pod "global-pull-secret-syncer-2z2wd" (UID: "bc73ab72-c095-4e0d-bbd9-b5296cadc187") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:33:15.199017 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197727    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-cnibin\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.199017 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197793    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-device-dir\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.199017 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197640    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-system-cni-dir\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.199017 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197898    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-os-release\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.199017 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197901    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.199017 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197711    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-host-run-k8s-cni-cncf-io\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.199017 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197961    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.199017 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197966    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.199017 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.197982    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-host-var-lib-cni-multus\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.199017 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198013    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-multus-socket-dir-parent\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.199017 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198099    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-sys-fs\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.199017 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198234    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/13b7a2b4-3185-4eae-98d8-855f2325e268-serviceca\") pod \"node-ca-4jm79\" (UID: \"13b7a2b4-3185-4eae-98d8-855f2325e268\") " pod="openshift-image-registry/node-ca-4jm79"
Aug 05 17:33:15.199017 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198289    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-cnibin\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.199017 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198317    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-host-var-lib-cni-bin\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.199566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198342    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-os-release\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.199566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198356    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06d69006-abdd-452a-8aba-5f50ec48a6c1-cni-binary-copy\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.199566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198366    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bc73ab72-c095-4e0d-bbd9-b5296cadc187-dbus\") pod \"global-pull-secret-syncer-2z2wd\" (UID: \"bc73ab72-c095-4e0d-bbd9-b5296cadc187\") " pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:15.199566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198390    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-cnibin\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.199566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198399    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06d69006-abdd-452a-8aba-5f50ec48a6c1-host-var-lib-cni-bin\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.199566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198423    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-os-release\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.199566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198436    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bc73ab72-c095-4e0d-bbd9-b5296cadc187-dbus\") pod \"global-pull-secret-syncer-2z2wd\" (UID: \"bc73ab72-c095-4e0d-bbd9-b5296cadc187\") " pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:15.199566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198449    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06d69006-abdd-452a-8aba-5f50ec48a6c1-multus-daemon-config\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.199566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198472    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq2gt\" (UniqueName: \"kubernetes.io/projected/fab79c82-8579-4ce6-9410-89fce50e5b70-kube-api-access-dq2gt\") pod \"network-metrics-daemon-x7hnn\" (UID: \"fab79c82-8579-4ce6-9410-89fce50e5b70\") " pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:15.199566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198498    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.199566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198533    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-etc-selinux\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.199566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198558    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.199566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198663    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-etc-selinux\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.199566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198699    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.199566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.198976    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.199566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.199558    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06d69006-abdd-452a-8aba-5f50ec48a6c1-multus-daemon-config\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.208937 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.208913    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjmzx\" (UniqueName: \"kubernetes.io/projected/91685c48-2cdf-4ccc-9d5c-49688e13d0eb-kube-api-access-hjmzx\") pod \"multus-additional-cni-plugins-42zkd\" (UID: \"91685c48-2cdf-4ccc-9d5c-49688e13d0eb\") " pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.209277 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:15.209255    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:33:15.209377 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:15.209284    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:33:15.209377 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:15.209297    2569 projected.go:196] Error preparing data for projected volume kube-api-access-xqqc4 for pod openshift-network-diagnostics/network-check-target-9vkxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:33:15.209377 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:15.209377    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4 podName:e538a505-e584-496e-9907-e2fb84f0e872 nodeName:}" failed. No retries permitted until 2026-08-05 17:33:15.70936026 +0000 UTC m=+3.062034850 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xqqc4" (UniqueName: "kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4") pod "network-check-target-9vkxl" (UID: "e538a505-e584-496e-9907-e2fb84f0e872") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:33:15.210211 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.210171    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbkhz\" (UniqueName: \"kubernetes.io/projected/13b7a2b4-3185-4eae-98d8-855f2325e268-kube-api-access-cbkhz\") pod \"node-ca-4jm79\" (UID: \"13b7a2b4-3185-4eae-98d8-855f2325e268\") " pod="openshift-image-registry/node-ca-4jm79"
Aug 05 17:33:15.210683 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.210643    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq2gt\" (UniqueName: \"kubernetes.io/projected/fab79c82-8579-4ce6-9410-89fce50e5b70-kube-api-access-dq2gt\") pod \"network-metrics-daemon-x7hnn\" (UID: \"fab79c82-8579-4ce6-9410-89fce50e5b70\") " pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:15.211019 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.210986    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-922kd\" (UniqueName: \"kubernetes.io/projected/06d69006-abdd-452a-8aba-5f50ec48a6c1-kube-api-access-922kd\") pod \"multus-vtq9n\" (UID: \"06d69006-abdd-452a-8aba-5f50ec48a6c1\") " pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.212561 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.212499    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5qpl\" (UniqueName: \"kubernetes.io/projected/77d69a26-bd5e-4d8b-b569-d0bd3c3f7653-kube-api-access-n5qpl\") pod \"aws-ebs-csi-driver-node-fl5sx\" (UID: \"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.212888 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.212870    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j5n6\" (UniqueName: \"kubernetes.io/projected/edf39462-7bb0-4c83-8979-c71a9df1caba-kube-api-access-9j5n6\") pod \"node-resolver-v8pbg\" (UID: \"edf39462-7bb0-4c83-8979-c71a9df1caba\") " pod="openshift-dns/node-resolver-v8pbg"
Aug 05 17:33:15.297339 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.297271    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gfl7r"
Aug 05 17:33:15.306257 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.306235    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dp52t"
Aug 05 17:33:15.318159 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.318136    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:15.324043 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.324026    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4tkxv"
Aug 05 17:33:15.332855 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.332839    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4jm79"
Aug 05 17:33:15.340110 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.340091    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx"
Aug 05 17:33:15.354792 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.354774    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vtq9n"
Aug 05 17:33:15.364494 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.364476    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-v8pbg"
Aug 05 17:33:15.371151 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.371130    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-42zkd"
Aug 05 17:33:15.384571 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.384551    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:33:15.700934 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.700854    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret\") pod \"global-pull-secret-syncer-2z2wd\" (UID: \"bc73ab72-c095-4e0d-bbd9-b5296cadc187\") " pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:15.700934 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.700925    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs\") pod \"network-metrics-daemon-x7hnn\" (UID: \"fab79c82-8579-4ce6-9410-89fce50e5b70\") " pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:15.701151 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:15.701031    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:33:15.701151 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:15.701041    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:33:15.701151 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:15.701115    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs podName:fab79c82-8579-4ce6-9410-89fce50e5b70 nodeName:}" failed. No retries permitted until 2026-08-05 17:33:16.701098573 +0000 UTC m=+4.053773155 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs") pod "network-metrics-daemon-x7hnn" (UID: "fab79c82-8579-4ce6-9410-89fce50e5b70") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:33:15.701151 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:15.701130    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret podName:bc73ab72-c095-4e0d-bbd9-b5296cadc187 nodeName:}" failed. No retries permitted until 2026-08-05 17:33:16.701124253 +0000 UTC m=+4.053798827 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret") pod "global-pull-secret-syncer-2z2wd" (UID: "bc73ab72-c095-4e0d-bbd9-b5296cadc187") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:33:15.802207 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:15.802164    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqqc4\" (UniqueName: \"kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4\") pod \"network-check-target-9vkxl\" (UID: \"e538a505-e584-496e-9907-e2fb84f0e872\") " pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:15.802363 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:15.802332    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:33:15.802363 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:15.802347    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:33:15.802363 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:15.802356    2569 projected.go:196] Error preparing data for projected volume kube-api-access-xqqc4 for pod openshift-network-diagnostics/network-check-target-9vkxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:33:15.802482 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:15.802408    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4 podName:e538a505-e584-496e-9907-e2fb84f0e872 nodeName:}" failed. No retries permitted until 2026-08-05 17:33:16.80239118 +0000 UTC m=+4.155065753 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xqqc4" (UniqueName: "kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4") pod "network-check-target-9vkxl" (UID: "e538a505-e584-496e-9907-e2fb84f0e872") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:33:16.044218 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:16.044076    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-04 17:28:15 +0000 UTC" deadline="2028-04-18 21:17:12.78101815 +0000 UTC"
Aug 05 17:33:16.044218 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:16.044135    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14931h43m56.736887687s"
Aug 05 17:33:16.144867 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:16.144839    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:16.144867 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:16.144868    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:16.145092 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:16.144954    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:16.145139 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:16.145120    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:16.207668 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:16.207642    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77d69a26_bd5e_4d8b_b569_d0bd3c3f7653.slice/crio-44b851bc964bce020f9b979a0a4cfd049a481bc6b50ed02e6bdd6c2832eee504 WatchSource:0}: Error finding container 44b851bc964bce020f9b979a0a4cfd049a481bc6b50ed02e6bdd6c2832eee504: Status 404 returned error can't find the container with id 44b851bc964bce020f9b979a0a4cfd049a481bc6b50ed02e6bdd6c2832eee504
Aug 05 17:33:16.222593 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:16.222568    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc34ee1d_5676_425e_b5f7_793ec5b14b64.slice/crio-2b5ac9b54bc63aa8f785814602444333b9c6e664cd007fda51569bdda9e16497 WatchSource:0}: Error finding container 2b5ac9b54bc63aa8f785814602444333b9c6e664cd007fda51569bdda9e16497: Status 404 returned error can't find the container with id 2b5ac9b54bc63aa8f785814602444333b9c6e664cd007fda51569bdda9e16497
Aug 05 17:33:16.226369 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:16.226286    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda043e58e_b56d_4a56_b92b_7fb8f70694bc.slice/crio-1b3ea7863eb1c1d752941a7b935a5f4100400e8308f053dbb4ef901936a1e69a WatchSource:0}: Error finding container 1b3ea7863eb1c1d752941a7b935a5f4100400e8308f053dbb4ef901936a1e69a: Status 404 returned error can't find the container with id 1b3ea7863eb1c1d752941a7b935a5f4100400e8308f053dbb4ef901936a1e69a
Aug 05 17:33:16.230043 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:16.229823    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91685c48_2cdf_4ccc_9d5c_49688e13d0eb.slice/crio-7c2454cb2ced3d467e1b0eb8da860b3a12243e6cab30b1beb9e0bf1e596cdfe9 WatchSource:0}: Error finding container 7c2454cb2ced3d467e1b0eb8da860b3a12243e6cab30b1beb9e0bf1e596cdfe9: Status 404 returned error can't find the container with id 7c2454cb2ced3d467e1b0eb8da860b3a12243e6cab30b1beb9e0bf1e596cdfe9
Aug 05 17:33:16.231384 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:16.231228    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6826678_5aa6_47c3_8cf8_ed7eb86a51c2.slice/crio-65d767c0f0434e5a1ad32f0358a0700876b40e7682001fd6f6ba29aaec740632 WatchSource:0}: Error finding container 65d767c0f0434e5a1ad32f0358a0700876b40e7682001fd6f6ba29aaec740632: Status 404 returned error can't find the container with id 65d767c0f0434e5a1ad32f0358a0700876b40e7682001fd6f6ba29aaec740632
Aug 05 17:33:16.233399 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:33:16.233363    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06d69006_abdd_452a_8aba_5f50ec48a6c1.slice/crio-adc1000b9c488026be3116b85fdc8345edd599de243226bf683671237d2f2a49 WatchSource:0}: Error finding container adc1000b9c488026be3116b85fdc8345edd599de243226bf683671237d2f2a49: Status 404 returned error can't find the container with id adc1000b9c488026be3116b85fdc8345edd599de243226bf683671237d2f2a49
Aug 05 17:33:16.710769 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:16.709110    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret\") pod \"global-pull-secret-syncer-2z2wd\" (UID: \"bc73ab72-c095-4e0d-bbd9-b5296cadc187\") " pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:16.710769 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:16.709420    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs\") pod \"network-metrics-daemon-x7hnn\" (UID: \"fab79c82-8579-4ce6-9410-89fce50e5b70\") " pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:16.710769 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:16.709546    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:33:16.710769 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:16.709606    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs podName:fab79c82-8579-4ce6-9410-89fce50e5b70 nodeName:}" failed. No retries permitted until 2026-08-05 17:33:18.709588911 +0000 UTC m=+6.062263488 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs") pod "network-metrics-daemon-x7hnn" (UID: "fab79c82-8579-4ce6-9410-89fce50e5b70") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:33:16.710769 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:16.709977    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:33:16.710769 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:16.710016    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret podName:bc73ab72-c095-4e0d-bbd9-b5296cadc187 nodeName:}" failed. No retries permitted until 2026-08-05 17:33:18.710005442 +0000 UTC m=+6.062680018 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret") pod "global-pull-secret-syncer-2z2wd" (UID: "bc73ab72-c095-4e0d-bbd9-b5296cadc187") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:33:16.809901 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:16.809831    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqqc4\" (UniqueName: \"kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4\") pod \"network-check-target-9vkxl\" (UID: \"e538a505-e584-496e-9907-e2fb84f0e872\") " pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:16.810039 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:16.809993    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:33:16.810039 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:16.810016    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:33:16.810039 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:16.810029    2569 projected.go:196] Error preparing data for projected volume kube-api-access-xqqc4 for pod openshift-network-diagnostics/network-check-target-9vkxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:33:16.810152 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:16.810087    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4 podName:e538a505-e584-496e-9907-e2fb84f0e872 nodeName:}" failed. No retries permitted until 2026-08-05 17:33:18.810068443 +0000 UTC m=+6.162743035 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xqqc4" (UniqueName: "kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4") pod "network-check-target-9vkxl" (UID: "e538a505-e584-496e-9907-e2fb84f0e872") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:33:17.057042 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:17.056962    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-04 17:28:15 +0000 UTC" deadline="2028-03-31 18:39:22.324131821 +0000 UTC"
Aug 05 17:33:17.057042 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:17.056993    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14497h6m5.267144172s"
Aug 05 17:33:17.145389 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:17.145061    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:17.145389 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:17.145237    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:17.177577 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:17.177531    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-42zkd" event={"ID":"91685c48-2cdf-4ccc-9d5c-49688e13d0eb","Type":"ContainerStarted","Data":"7c2454cb2ced3d467e1b0eb8da860b3a12243e6cab30b1beb9e0bf1e596cdfe9"}
Aug 05 17:33:17.181396 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:17.181269    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4" event={"ID":"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1","Type":"ContainerStarted","Data":"5accff503e7fd24a611bd3904390209c4f3649d6a2b1aa1f097fa944082d7c40"}
Aug 05 17:33:17.197523 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:17.197440    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-v8pbg" event={"ID":"edf39462-7bb0-4c83-8979-c71a9df1caba","Type":"ContainerStarted","Data":"e677775ee5e1f9fb49eb029448bf8c5078689558b2c25918890577d83fdd8266"}
Aug 05 17:33:17.208822 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:17.208705    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dp52t" event={"ID":"fc34ee1d-5676-425e-b5f7-793ec5b14b64","Type":"ContainerStarted","Data":"2b5ac9b54bc63aa8f785814602444333b9c6e664cd007fda51569bdda9e16497"}
Aug 05 17:33:17.213501 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:17.212660    2569 generic.go:356] "Generic (PLEG): container finished" podID="4b12f1453a65ccfd781f199e0071a899" containerID="eb1af5be34bd7f6990101d00b48709c50c2c81e82af9e3c01c32c044235b363c" exitCode=0
Aug 05 17:33:17.213501 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:17.212738    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal" event={"ID":"4b12f1453a65ccfd781f199e0071a899","Type":"ContainerDied","Data":"eb1af5be34bd7f6990101d00b48709c50c2c81e82af9e3c01c32c044235b363c"}
Aug 05 17:33:17.226457 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:17.226320    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gfl7r" event={"ID":"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2","Type":"ContainerStarted","Data":"65d767c0f0434e5a1ad32f0358a0700876b40e7682001fd6f6ba29aaec740632"}
Aug 05 17:33:17.242875 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:17.242848    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vtq9n" event={"ID":"06d69006-abdd-452a-8aba-5f50ec48a6c1","Type":"ContainerStarted","Data":"adc1000b9c488026be3116b85fdc8345edd599de243226bf683671237d2f2a49"}
Aug 05 17:33:17.256755 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:17.256726    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4tkxv" event={"ID":"a043e58e-b56d-4a56-b92b-7fb8f70694bc","Type":"ContainerStarted","Data":"1b3ea7863eb1c1d752941a7b935a5f4100400e8308f053dbb4ef901936a1e69a"}
Aug 05 17:33:17.266155 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:17.266081    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4jm79" event={"ID":"13b7a2b4-3185-4eae-98d8-855f2325e268","Type":"ContainerStarted","Data":"34d419a9fd8aa00a2dc55d5f5709de99cc0e2d226953320e394d81bedcb317ef"}
Aug 05 17:33:17.278587 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:17.278481    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx" event={"ID":"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653","Type":"ContainerStarted","Data":"44b851bc964bce020f9b979a0a4cfd049a481bc6b50ed02e6bdd6c2832eee504"}
Aug 05 17:33:17.286104 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:17.286076    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-77.ec2.internal" event={"ID":"beae357a1d1b622108905e09a0f5f3aa","Type":"ContainerStarted","Data":"664df782454896237a92853c235dfbbbff7bfe611424379f37b7ac1288cc9972"}
Aug 05 17:33:18.145492 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:18.144813    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:18.145492 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:18.144938    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:18.145492 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:18.145322    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:18.145492 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:18.145410    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:18.308324 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:18.308284    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal" event={"ID":"4b12f1453a65ccfd781f199e0071a899","Type":"ContainerStarted","Data":"7b0b9d45097ac757d9166e6111bf117ff0db98d3402a20b76b07d887646ee6cf"}
Aug 05 17:33:18.329572 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:18.329517    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-77.ec2.internal" podStartSLOduration=4.329497605 podStartE2EDuration="4.329497605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:33:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:33:17.307452043 +0000 UTC m=+4.660126640" watchObservedRunningTime="2026-08-05 17:33:18.329497605 +0000 UTC m=+5.682172204"
Aug 05 17:33:18.725368 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:18.725335    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret\") pod \"global-pull-secret-syncer-2z2wd\" (UID: \"bc73ab72-c095-4e0d-bbd9-b5296cadc187\") " pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:18.725561 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:18.725402    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs\") pod \"network-metrics-daemon-x7hnn\" (UID: \"fab79c82-8579-4ce6-9410-89fce50e5b70\") " pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:18.725561 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:18.725548    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:33:18.725676 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:18.725605    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs podName:fab79c82-8579-4ce6-9410-89fce50e5b70 nodeName:}" failed. No retries permitted until 2026-08-05 17:33:22.725587249 +0000 UTC m=+10.078261823 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs") pod "network-metrics-daemon-x7hnn" (UID: "fab79c82-8579-4ce6-9410-89fce50e5b70") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:33:18.726064 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:18.726038    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:33:18.726160 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:18.726098    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret podName:bc73ab72-c095-4e0d-bbd9-b5296cadc187 nodeName:}" failed. No retries permitted until 2026-08-05 17:33:22.726082958 +0000 UTC m=+10.078757547 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret") pod "global-pull-secret-syncer-2z2wd" (UID: "bc73ab72-c095-4e0d-bbd9-b5296cadc187") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:33:18.826303 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:18.825721    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqqc4\" (UniqueName: \"kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4\") pod \"network-check-target-9vkxl\" (UID: \"e538a505-e584-496e-9907-e2fb84f0e872\") " pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:18.826303 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:18.825890    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:33:18.826303 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:18.825914    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:33:18.826303 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:18.825928    2569 projected.go:196] Error preparing data for projected volume kube-api-access-xqqc4 for pod openshift-network-diagnostics/network-check-target-9vkxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:33:18.826303 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:18.825978    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4 podName:e538a505-e584-496e-9907-e2fb84f0e872 nodeName:}" failed. No retries permitted until 2026-08-05 17:33:22.825961735 +0000 UTC m=+10.178636309 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xqqc4" (UniqueName: "kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4") pod "network-check-target-9vkxl" (UID: "e538a505-e584-496e-9907-e2fb84f0e872") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:33:19.145803 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:19.145288    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:19.145803 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:19.145422    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:20.144359 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:20.144327    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:20.144535 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:20.144438    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:20.144535 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:20.144500    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:20.144649 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:20.144587    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:21.144595 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:21.144558    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:21.145014 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:21.144705    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:22.144370 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:22.144338    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:22.144562 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:22.144458    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:22.144845 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:22.144826    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:22.145262 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:22.144915    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:22.757608 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:22.757571    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs\") pod \"network-metrics-daemon-x7hnn\" (UID: \"fab79c82-8579-4ce6-9410-89fce50e5b70\") " pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:22.757799 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:22.757639    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret\") pod \"global-pull-secret-syncer-2z2wd\" (UID: \"bc73ab72-c095-4e0d-bbd9-b5296cadc187\") " pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:22.757799 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:22.757744    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:33:22.757799 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:22.757765    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:33:22.757968 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:22.757818    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs podName:fab79c82-8579-4ce6-9410-89fce50e5b70 nodeName:}" failed. No retries permitted until 2026-08-05 17:33:30.757797935 +0000 UTC m=+18.110472519 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs") pod "network-metrics-daemon-x7hnn" (UID: "fab79c82-8579-4ce6-9410-89fce50e5b70") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:33:22.757968 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:22.757835    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret podName:bc73ab72-c095-4e0d-bbd9-b5296cadc187 nodeName:}" failed. No retries permitted until 2026-08-05 17:33:30.757828487 +0000 UTC m=+18.110503062 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret") pod "global-pull-secret-syncer-2z2wd" (UID: "bc73ab72-c095-4e0d-bbd9-b5296cadc187") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:33:22.858073 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:22.858030    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqqc4\" (UniqueName: \"kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4\") pod \"network-check-target-9vkxl\" (UID: \"e538a505-e584-496e-9907-e2fb84f0e872\") " pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:22.858324 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:22.858213    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:33:22.858324 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:22.858229    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:33:22.858324 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:22.858239    2569 projected.go:196] Error preparing data for projected volume kube-api-access-xqqc4 for pod openshift-network-diagnostics/network-check-target-9vkxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:33:22.858324 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:22.858294    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4 podName:e538a505-e584-496e-9907-e2fb84f0e872 nodeName:}" failed. No retries permitted until 2026-08-05 17:33:30.858280485 +0000 UTC m=+18.210955074 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xqqc4" (UniqueName: "kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4") pod "network-check-target-9vkxl" (UID: "e538a505-e584-496e-9907-e2fb84f0e872") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:33:23.145381 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:23.145305    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:23.145844 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:23.145449    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:24.144264 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:24.144230    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:24.144450 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:24.144238    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:24.144450 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:24.144356    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:24.144450 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:24.144404    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:25.144803 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:25.144775    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:25.145160 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:25.144914    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:26.144646 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:26.144613    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:26.144839 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:26.144629    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:26.144839 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:26.144737    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:26.145264 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:26.144848    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:27.144227 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:27.144201    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:27.144376 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:27.144303    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:28.144700 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:28.144665    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:28.145153 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:28.144669    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:28.145153 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:28.144788    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:28.145153 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:28.144859    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:29.145274 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:29.145243    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:29.145692 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:29.145361    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:30.144720 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:30.144250    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:30.144720 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:30.144269    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:30.144720 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:30.144374    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:30.144720 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:30.144505    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:30.821454 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:30.821427    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs\") pod \"network-metrics-daemon-x7hnn\" (UID: \"fab79c82-8579-4ce6-9410-89fce50e5b70\") " pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:30.821824 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:30.821473    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret\") pod \"global-pull-secret-syncer-2z2wd\" (UID: \"bc73ab72-c095-4e0d-bbd9-b5296cadc187\") " pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:30.821824 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:30.821567    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:33:30.821824 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:30.821567    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:33:30.821824 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:30.821633    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret podName:bc73ab72-c095-4e0d-bbd9-b5296cadc187 nodeName:}" failed. No retries permitted until 2026-08-05 17:33:46.821612796 +0000 UTC m=+34.174287373 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret") pod "global-pull-secret-syncer-2z2wd" (UID: "bc73ab72-c095-4e0d-bbd9-b5296cadc187") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:33:30.821824 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:30.821651    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs podName:fab79c82-8579-4ce6-9410-89fce50e5b70 nodeName:}" failed. No retries permitted until 2026-08-05 17:33:46.821643702 +0000 UTC m=+34.174318274 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs") pod "network-metrics-daemon-x7hnn" (UID: "fab79c82-8579-4ce6-9410-89fce50e5b70") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:33:30.921791 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:30.921756    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqqc4\" (UniqueName: \"kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4\") pod \"network-check-target-9vkxl\" (UID: \"e538a505-e584-496e-9907-e2fb84f0e872\") " pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:30.921948 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:30.921916    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:33:30.921948 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:30.921939    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:33:30.921948 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:30.921950    2569 projected.go:196] Error preparing data for projected volume kube-api-access-xqqc4 for pod openshift-network-diagnostics/network-check-target-9vkxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:33:30.922081 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:30.922008    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4 podName:e538a505-e584-496e-9907-e2fb84f0e872 nodeName:}" failed. No retries permitted until 2026-08-05 17:33:46.921991368 +0000 UTC m=+34.274665945 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xqqc4" (UniqueName: "kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4") pod "network-check-target-9vkxl" (UID: "e538a505-e584-496e-9907-e2fb84f0e872") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:33:31.144747 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:31.144667    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:31.144883 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:31.144796    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:32.144218 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:32.144172    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:32.144592 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:32.144281    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:32.144592 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:32.144575    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:32.144695 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:32.144677    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:33.145336 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:33.145303    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:33.145768 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:33.145399    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:34.144670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:34.144639    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:34.144849 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:34.144676    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:34.144849 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:34.144753    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:34.144956 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:34.144906    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:35.147560 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.147382    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:35.149066 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:35.148428    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:35.352200 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.351798    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-v8pbg" event={"ID":"edf39462-7bb0-4c83-8979-c71a9df1caba","Type":"ContainerStarted","Data":"4ec22d399145273d8316f3cb7fa7e01b2ee75b3fc21e24cab61b05421bdf0bea"}
Aug 05 17:33:35.353521 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.353492    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gfl7r" event={"ID":"f6826678-5aa6-47c3-8cf8-ed7eb86a51c2","Type":"ContainerStarted","Data":"ddf0d9daaa23d6747da14f3a4525dfc73fb4c155a95133083a9caf11c5effefc"}
Aug 05 17:33:35.355103 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.355074    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vtq9n" event={"ID":"06d69006-abdd-452a-8aba-5f50ec48a6c1","Type":"ContainerStarted","Data":"4197b5616c4191196e67eb84bb21fb47ef8b39edf27efff8f8c4a7be7e9ac240"}
Aug 05 17:33:35.356842 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.356820    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4tkxv" event={"ID":"a043e58e-b56d-4a56-b92b-7fb8f70694bc","Type":"ContainerStarted","Data":"68bae15359008301c99fd6595b461086c952276dca70019e43fae3f3f88fe918"}
Aug 05 17:33:35.358547 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.358518    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4jm79" event={"ID":"13b7a2b4-3185-4eae-98d8-855f2325e268","Type":"ContainerStarted","Data":"9907e84998b1e5d7c8ad2a7443bb58e8d06ac0d7ad78aff931651aba2d54c1cc"}
Aug 05 17:33:35.360410 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.360387    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx" event={"ID":"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653","Type":"ContainerStarted","Data":"76971d1351f6fa77b85bb03b3b54127d37c60d520010422942d397a8e3e8259a"}
Aug 05 17:33:35.361813 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.361790    2569 generic.go:356] "Generic (PLEG): container finished" podID="91685c48-2cdf-4ccc-9d5c-49688e13d0eb" containerID="ced20b29abe3251b556a450626a27e84c3f473bf5bc2e6dba7788a56009070bf" exitCode=0
Aug 05 17:33:35.361988 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.361872    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-42zkd" event={"ID":"91685c48-2cdf-4ccc-9d5c-49688e13d0eb","Type":"ContainerDied","Data":"ced20b29abe3251b556a450626a27e84c3f473bf5bc2e6dba7788a56009070bf"}
Aug 05 17:33:35.364292 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.364274    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4" event={"ID":"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1","Type":"ContainerStarted","Data":"a51d5efcab292bb8bab6d3125c87f6c41520aed9be11b86ee88e2720b0325fe5"}
Aug 05 17:33:35.364364 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.364327    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4" event={"ID":"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1","Type":"ContainerStarted","Data":"3ab2f2d335804af06d41c28032703e2deeb970da2aec83c99336400286c03779"}
Aug 05 17:33:35.364403 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.364368    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4" event={"ID":"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1","Type":"ContainerStarted","Data":"d5f60e6c790d91fbfa8d5eef009e29e479907ea805af5a8eb1ee885299a7bbd7"}
Aug 05 17:33:35.375853 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.375817    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-v8pbg" podStartSLOduration=3.5980818599999997 podStartE2EDuration="22.375803154s" totalImagesPullingTime="18.777721294s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:33:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:33:35.375430495 +0000 UTC m=+22.728105088" watchObservedRunningTime="2026-08-05 17:33:35.375803154 +0000 UTC m=+22.728477748"
Aug 05 17:33:35.375960 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.375886    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-77.ec2.internal" podStartSLOduration=21.375880054 podStartE2EDuration="21.375880054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:33:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:33:18.330685864 +0000 UTC m=+5.683360459" watchObservedRunningTime="2026-08-05 17:33:35.375880054 +0000 UTC m=+22.728554653"
Aug 05 17:33:35.395940 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.395904    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4tkxv" podStartSLOduration=7.318893563 podStartE2EDuration="22.395890397s" totalImagesPullingTime="15.076996834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:33:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:33:35.395828811 +0000 UTC m=+22.748503406" watchObservedRunningTime="2026-08-05 17:33:35.395890397 +0000 UTC m=+22.748564990"
Aug 05 17:33:35.417241 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.417201    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4jm79" podStartSLOduration=3.637797735 podStartE2EDuration="22.417168729s" totalImagesPullingTime="18.779370994s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:33:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:33:35.416977428 +0000 UTC m=+22.769652004" watchObservedRunningTime="2026-08-05 17:33:35.417168729 +0000 UTC m=+22.769843323"
Aug 05 17:33:35.463402 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.463369    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vtq9n" podStartSLOduration=3.6389839 podStartE2EDuration="22.463357872s" totalImagesPullingTime="18.824373972s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:33:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:33:35.439432881 +0000 UTC m=+22.792107475" watchObservedRunningTime="2026-08-05 17:33:35.463357872 +0000 UTC m=+22.816032465"
Aug 05 17:33:35.487869 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.487835    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gfl7r" podStartSLOduration=3.699682816 podStartE2EDuration="22.487824735s" totalImagesPullingTime="18.788141919s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:33:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:33:35.487304437 +0000 UTC m=+22.839979028" watchObservedRunningTime="2026-08-05 17:33:35.487824735 +0000 UTC m=+22.840499329"
Aug 05 17:33:35.497539 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.497516    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:35.575944 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.575910    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:35.641737 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:35.641715    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:36.144993 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:36.144951    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:36.144993 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:36.144984    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:36.145201 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:36.145060    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:36.145201 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:36.145122    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:36.370287 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:36.369873    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4" event={"ID":"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1","Type":"ContainerStarted","Data":"dfd6b8c86a6fe4ef0f02e85eaf0fd9ac49088c68709d83eca07890014ce70de6"}
Aug 05 17:33:36.370287 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:36.369946    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4" event={"ID":"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1","Type":"ContainerStarted","Data":"3ec11f9288acb4856853e1e6cf58f40549fe08329a5b208274f7e0a26b390827"}
Aug 05 17:33:36.370287 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:36.369992    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4" event={"ID":"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1","Type":"ContainerStarted","Data":"6972bb872fb3070243d815ed91f9cba572c7d0e50f56396eff3f3249bafe6a11"}
Aug 05 17:33:36.573432 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:36.573395    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Aug 05 17:33:36.810953 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:36.810893    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4tkxv"
Aug 05 17:33:36.811424 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:36.811410    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4tkxv"
Aug 05 17:33:37.068473 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:37.068330    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-05T17:33:36.573411548Z","UUID":"22d544c0-0d41-4367-bfc7-dedbe29c7215","Handler":null,"Name":"","Endpoint":""}
Aug 05 17:33:37.069914 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:37.069894    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Aug 05 17:33:37.070047 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:37.069928    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Aug 05 17:33:37.144857 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:37.144832    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:37.145010 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:37.144972    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:37.373846 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:37.373772    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx" event={"ID":"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653","Type":"ContainerStarted","Data":"05e1e012b16d9f638dcae8faa05347e6cd04db711fd0d3e5fe6125f1eb5d3964"}
Aug 05 17:33:37.376063 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:37.376032    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dp52t" event={"ID":"fc34ee1d-5676-425e-b5f7-793ec5b14b64","Type":"ContainerStarted","Data":"37df07b6cb8b52a7a2ef2e8f77b600f8ed4db20ccbc7e13b72a226f5977c25af"}
Aug 05 17:33:37.376407 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:37.376217    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4tkxv"
Aug 05 17:33:37.377337 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:37.376859    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4tkxv"
Aug 05 17:33:37.396367 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:37.396319    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dp52t" podStartSLOduration=9.316096837 podStartE2EDuration="24.396302589s" totalImagesPullingTime="15.080205752s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:33:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:33:37.395799315 +0000 UTC m=+24.748473908" watchObservedRunningTime="2026-08-05 17:33:37.396302589 +0000 UTC m=+24.748977184"
Aug 05 17:33:37.535214 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:37.535135    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:37.635798 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:37.635775    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:37.740583 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:37.740525    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:38.145136 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:38.145110    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:38.145581 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:38.145549    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:38.148636 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:38.148026    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:38.148636 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:38.148162    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:38.380612 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:38.380575    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx" event={"ID":"77d69a26-bd5e-4d8b-b569-d0bd3c3f7653","Type":"ContainerStarted","Data":"d100e719c71d9fed3156c224166cab90113bdaae74ba1a334f0d3968985b1e82"}
Aug 05 17:33:38.384277 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:38.384240    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4" event={"ID":"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1","Type":"ContainerStarted","Data":"e902a77fc232c9ab768f154f6104e942bd0d673f31bff89a2a108accf2564e4e"}
Aug 05 17:33:38.420829 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:38.420764    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fl5sx" podStartSLOduration=4.248341425 podStartE2EDuration="25.420751402s" totalImagesPullingTime="21.172409977s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:33:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:33:38.410027585 +0000 UTC m=+25.762702179" watchObservedRunningTime="2026-08-05 17:33:38.420751402 +0000 UTC m=+25.773426022"
Aug 05 17:33:38.702165 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:38.702087    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:38.799580 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:38.799551    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:39.147762 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:39.147561    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:39.147927 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:39.147810    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:39.452997 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:39.452911    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:39.565451 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:39.565413    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:39.702350 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:39.702321    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:39.781504 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:39.781478    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:39.881827 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:39.881793    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:40.144277 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:40.144194    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:40.144277 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:40.144215    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:40.144468 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:40.144316    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:40.144468 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:40.144458    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:40.309381 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:40.309348    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:40.435872 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:40.435602    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:40.969900 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:40.969834    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-v8pbg_edf39462-7bb0-4c83-8979-c71a9df1caba/dns-node-resolver/0.log"
Aug 05 17:33:41.145298 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:41.145126    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:41.145457 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:41.145415    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:41.392959 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:41.392882    2569 generic.go:356] "Generic (PLEG): container finished" podID="91685c48-2cdf-4ccc-9d5c-49688e13d0eb" containerID="56572d42eb11bd2b8117e3cbe75184c1773ba6541f3d0983a51a642d0a18de97" exitCode=0
Aug 05 17:33:41.392959 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:41.392927    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-42zkd" event={"ID":"91685c48-2cdf-4ccc-9d5c-49688e13d0eb","Type":"ContainerDied","Data":"56572d42eb11bd2b8117e3cbe75184c1773ba6541f3d0983a51a642d0a18de97"}
Aug 05 17:33:41.396690 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:41.396664    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4" event={"ID":"5b61ea1d-7181-4a3c-8bd3-989ce74d20a1","Type":"ContainerStarted","Data":"f567212f68fd1daec466045f14dd7a0e900ee4b0395bf85a448714535a3c5387"}
Aug 05 17:33:41.396888 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:41.396871    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:41.396976 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:41.396943    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:41.413335 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:41.413311    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:41.466900 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:41.466857    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4" podStartSLOduration=9.629571459 podStartE2EDuration="28.466845398s" totalImagesPullingTime="18.837273939s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:33:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:33:41.465223643 +0000 UTC m=+28.817898236" watchObservedRunningTime="2026-08-05 17:33:41.466845398 +0000 UTC m=+28.819519991"
Aug 05 17:33:41.766859 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:41.766772    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4jm79_13b7a2b4-3185-4eae-98d8-855f2325e268/node-ca/0.log"
Aug 05 17:33:42.036604 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:42.036529    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:42.144669 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:42.144641    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:42.144808 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:42.144671    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:42.144808 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:42.144740    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:42.144915 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:42.144888    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:42.175110 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:42.175074    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:42.401094 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:42.401066    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-42zkd" event={"ID":"91685c48-2cdf-4ccc-9d5c-49688e13d0eb","Type":"ContainerStarted","Data":"9eb0d004eed321abfe81731201a3483c4216c820672b6f401201ca3890fe5b0a"}
Aug 05 17:33:42.401643 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:42.401615    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:42.418082 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:42.418058    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:33:42.555272 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:42.555210    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:42.660709 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:42.660683    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:42.675661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:42.675635    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9vkxl"]
Aug 05 17:33:42.675787 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:42.675732    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:42.675841 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:42.675812    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:42.681655 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:42.681634    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2z2wd"]
Aug 05 17:33:42.681754 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:42.681736    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:42.681838 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:42.681818    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:42.682300 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:42.682280    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x7hnn"]
Aug 05 17:33:42.682404 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:42.682387    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:42.682494 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:42.682478    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:43.100162 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:43.100129    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:43.405371 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:43.405285    2569 generic.go:356] "Generic (PLEG): container finished" podID="91685c48-2cdf-4ccc-9d5c-49688e13d0eb" containerID="9eb0d004eed321abfe81731201a3483c4216c820672b6f401201ca3890fe5b0a" exitCode=0
Aug 05 17:33:43.405516 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:43.405365    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-42zkd" event={"ID":"91685c48-2cdf-4ccc-9d5c-49688e13d0eb","Type":"ContainerDied","Data":"9eb0d004eed321abfe81731201a3483c4216c820672b6f401201ca3890fe5b0a"}
Aug 05 17:33:43.592426 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:43.592395    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:43.773289 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:43.773216    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:43.900205 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:43.900163    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:44.041766 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:44.041662    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:44.144800 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:44.144770    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:44.145120 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:44.144873    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:44.145120 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:44.144877    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:44.145120 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:44.144908    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:44.145120 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:44.145020    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:44.145120 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:44.145113    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:44.287346 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:44.287307    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:45.115923 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:45.115745    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:45.217052 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:45.217018    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:45.412635 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:45.412563    2569 generic.go:356] "Generic (PLEG): container finished" podID="91685c48-2cdf-4ccc-9d5c-49688e13d0eb" containerID="66f92f7ec5b9d22a93383aff22285fe8dd06cfa744d24442cb55df19ca604cb1" exitCode=0
Aug 05 17:33:45.412635 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:45.412608    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-42zkd" event={"ID":"91685c48-2cdf-4ccc-9d5c-49688e13d0eb","Type":"ContainerDied","Data":"66f92f7ec5b9d22a93383aff22285fe8dd06cfa744d24442cb55df19ca604cb1"}
Aug 05 17:33:45.495373 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:45.495348    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:45.696846 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:45.696766    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:45.934311 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:45.934277    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:46.034647 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:46.034572    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:46.144418 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:46.144383    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:46.144418 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:46.144421    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:46.144636 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:46.144391    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:46.144636 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:46.144493    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:46.144636 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:46.144580    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:46.144749 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:46.144655    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:46.146059 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:46.146034    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:46.324675 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:46.324645    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:46.835711 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:46.835670    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret\") pod \"global-pull-secret-syncer-2z2wd\" (UID: \"bc73ab72-c095-4e0d-bbd9-b5296cadc187\") " pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:46.835894 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:46.835735    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs\") pod \"network-metrics-daemon-x7hnn\" (UID: \"fab79c82-8579-4ce6-9410-89fce50e5b70\") " pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:46.835894 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:46.835818    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:33:46.835894 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:46.835873    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:33:46.836033 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:46.835877    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret podName:bc73ab72-c095-4e0d-bbd9-b5296cadc187 nodeName:}" failed. No retries permitted until 2026-08-05 17:34:18.835862472 +0000 UTC m=+66.188537045 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret") pod "global-pull-secret-syncer-2z2wd" (UID: "bc73ab72-c095-4e0d-bbd9-b5296cadc187") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:33:46.836033 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:46.835962    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs podName:fab79c82-8579-4ce6-9410-89fce50e5b70 nodeName:}" failed. No retries permitted until 2026-08-05 17:34:18.835944314 +0000 UTC m=+66.188618890 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs") pod "network-metrics-daemon-x7hnn" (UID: "fab79c82-8579-4ce6-9410-89fce50e5b70") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:33:46.936621 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:46.936589    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqqc4\" (UniqueName: \"kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4\") pod \"network-check-target-9vkxl\" (UID: \"e538a505-e584-496e-9907-e2fb84f0e872\") " pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:46.936777 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:46.936748    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:33:46.936777 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:46.936765    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:33:46.936882 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:46.936779    2569 projected.go:196] Error preparing data for projected volume kube-api-access-xqqc4 for pod openshift-network-diagnostics/network-check-target-9vkxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:33:46.936882 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:46.936844    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4 podName:e538a505-e584-496e-9907-e2fb84f0e872 nodeName:}" failed. No retries permitted until 2026-08-05 17:34:18.93682742 +0000 UTC m=+66.289502005 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xqqc4" (UniqueName: "kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4") pod "network-check-target-9vkxl" (UID: "e538a505-e584-496e-9907-e2fb84f0e872") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:33:47.293489 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:47.293411    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:47.394414 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:47.394379    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:47.561110 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:47.561024    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:47.723100 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:47.723070    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:47.940244 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:47.940198    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:48.081946 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:48.081915    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:48.144833 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:48.144803    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:48.145018 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:48.144900    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:48.145082 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:48.145012    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:48.145082 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:48.144906    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:48.145082 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:48.144903    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:48.145216 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:48.145121    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:48.184790 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:48.182899    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:48.283861 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:48.283777    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:49.283623 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:49.283586    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:49.573448 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:49.573411    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:49.672774 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:49.672741    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:49.747532 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:49.747499    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:49.873803 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:49.873729    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:50.083803 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:50.083735    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:50.145042 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:50.144949    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:50.145042 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:50.145008    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:50.145042 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:50.145009    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:50.145309 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:50.145095    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:50.145309 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:50.145214    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:50.145309 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:50.145292    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:50.483041 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:50.482964    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:51.072626 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:51.072588    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:51.430045 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:51.430008    2569 generic.go:356] "Generic (PLEG): container finished" podID="91685c48-2cdf-4ccc-9d5c-49688e13d0eb" containerID="ca0a70387753e8e139132a83f5e6d7900a5b45e7e23ea0cd16d8d2ec511cf693" exitCode=0
Aug 05 17:33:51.430215 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:51.430069    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-42zkd" event={"ID":"91685c48-2cdf-4ccc-9d5c-49688e13d0eb","Type":"ContainerDied","Data":"ca0a70387753e8e139132a83f5e6d7900a5b45e7e23ea0cd16d8d2ec511cf693"}
Aug 05 17:33:51.477488 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:51.477457    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:51.871812 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:51.871788    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:52.073199 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:52.073134    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:52.145098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:52.145029    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:52.145273 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:52.145135    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:52.145273 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:52.145162    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:52.145273 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:52.145163    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:52.145273 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:52.145243    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:52.145487 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:52.145347    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:52.434430 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:52.434359    2569 generic.go:356] "Generic (PLEG): container finished" podID="91685c48-2cdf-4ccc-9d5c-49688e13d0eb" containerID="c6aa5b968ddf1c70ef5e54b9acbbfdddb8e14d725fad31fbce179f2c4468228c" exitCode=0
Aug 05 17:33:52.434430 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:52.434409    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-42zkd" event={"ID":"91685c48-2cdf-4ccc-9d5c-49688e13d0eb","Type":"ContainerDied","Data":"c6aa5b968ddf1c70ef5e54b9acbbfdddb8e14d725fad31fbce179f2c4468228c"}
Aug 05 17:33:52.480045 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:52.479974    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:53.308101 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:53.308059    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:53.416436 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:53.416402    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:53.442842 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:53.442813    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-42zkd" event={"ID":"91685c48-2cdf-4ccc-9d5c-49688e13d0eb","Type":"ContainerStarted","Data":"d6e8ba7dbd4d0bbd915202809bba17e904f88180effa9fb4ce89bfc7f7777da8"}
Aug 05 17:33:53.473413 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:53.473363    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-42zkd" podStartSLOduration=9.257230562 podStartE2EDuration="40.47334831s" totalImagesPullingTime="31.216117748s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:33:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:33:53.472104264 +0000 UTC m=+40.824778859" watchObservedRunningTime="2026-08-05 17:33:53.47334831 +0000 UTC m=+40.826022903"
Aug 05 17:33:53.673789 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:53.673763    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:53.874761 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:53.874730    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:54.071408 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:54.071379    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:54.144674 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:54.144640    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:54.144860 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:54.144640    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:54.144860 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:54.144763    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:54.144860 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:54.144640    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:54.144860 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:54.144827    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:54.145071 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:54.144899    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:54.268438 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:54.268391    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:55.294076 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:55.294049    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:56.145072 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:56.145045    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:56.145269 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:56.145045    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:56.145269 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:56.145137    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:56.145269 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:56.145045    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:56.145269 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:56.145232    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:56.145497 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:56.145331    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:57.812714 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:57.812674    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:57.911411 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:57.911383    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:58.015622 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:58.015597    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:58.145191 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:58.145157    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:33:58.145319 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:58.145159    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:33:58.145319 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:58.145255    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:33:58.145392 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:58.145164    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:33:58.145392 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:58.145333    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:33:58.145460 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:33:58.145393    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:33:58.161822 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:58.161800    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:58.495765 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:58.495687    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:58.607031 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:58.607005    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:58.883821 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:58.883799    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:58.966522 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:58.966502    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:59.505874 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:59.505848    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:59.737943 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:33:59.737916    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:00.048523 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:00.048496    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:00.144606 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:00.144582    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:34:00.144734 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:00.144620    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:34:00.144734 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:34:00.144720    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:34:00.144841 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:34:00.144791    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:34:00.144841 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:00.144806    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:34:00.144942 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:34:00.144862    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:34:00.146335 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:00.146314    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:00.545282 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:00.545250    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:00.631605 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:00.631574    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:02.144443 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:02.144413    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:34:02.144804 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:02.144538    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:34:02.144804 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:02.144534    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:34:02.144804 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:34:02.144538    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:34:02.144804 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:34:02.144608    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:34:02.144804 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:34:02.144674    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:34:04.144668 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:04.144640    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:34:04.145109 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:04.144719    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:34:04.145109 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:34:04.144724    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:34:04.145109 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:04.144742    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:34:04.145109 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:34:04.144789    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:34:04.145109 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:34:04.144905    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:34:04.512074 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:04.511992    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:04.582326 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:04.582279    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:06.145141 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:06.145110    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:34:06.145609 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:06.145207    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:34:06.145609 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:34:06.145211    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkxl" podUID="e538a505-e584-496e-9907-e2fb84f0e872"
Aug 05 17:34:06.145609 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:34:06.145280    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2wd" podUID="bc73ab72-c095-4e0d-bbd9-b5296cadc187"
Aug 05 17:34:06.145609 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:06.145296    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:34:06.145609 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:34:06.145387    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7hnn" podUID="fab79c82-8579-4ce6-9410-89fce50e5b70"
Aug 05 17:34:06.392892 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:06.392865    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:06.489352 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:06.489290    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:06.595052 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:06.595026    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:06.691336 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:06.691313    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:06.787315 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:06.787257    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:06.942775 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:06.942755    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-77.ec2.internal" event="NodeReady"
Aug 05 17:34:06.942869 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:06.942851    2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Aug 05 17:34:07.089383 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.089364    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ntkvz"]
Aug 05 17:34:07.110116 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.110096    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ntkvz"
Aug 05 17:34:07.114001 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.113977    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-59qnw\""
Aug 05 17:34:07.114123 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.114079    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Aug 05 17:34:07.114300 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.114282    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ntkvz"]
Aug 05 17:34:07.115810 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.115794    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Aug 05 17:34:07.201604 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.201587    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-czglh"]
Aug 05 17:34:07.221795 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.221772    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-s79s8"]
Aug 05 17:34:07.221891 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.221878    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-czglh"
Aug 05 17:34:07.226950 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.226930    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Aug 05 17:34:07.227155 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.227081    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Aug 05 17:34:07.228366 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.228351    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xvqv4\""
Aug 05 17:34:07.228569 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.228557    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Aug 05 17:34:07.245780 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.245762    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s79s8"]
Aug 05 17:34:07.245870 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.245794    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-czglh"]
Aug 05 17:34:07.245911 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.245866    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s79s8"
Aug 05 17:34:07.250948 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.250907    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Aug 05 17:34:07.250948 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.250926    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Aug 05 17:34:07.251409 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.251389    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Aug 05 17:34:07.251946 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.251934    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Aug 05 17:34:07.251998 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.251937    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-b9t6s\""
Aug 05 17:34:07.290656 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.290629    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b92828f0-7eb1-4149-acad-b21d6845bf04-tmp-dir\") pod \"dns-default-ntkvz\" (UID: \"b92828f0-7eb1-4149-acad-b21d6845bf04\") " pod="openshift-dns/dns-default-ntkvz"
Aug 05 17:34:07.290749 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.290716    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b92828f0-7eb1-4149-acad-b21d6845bf04-metrics-tls\") pod \"dns-default-ntkvz\" (UID: \"b92828f0-7eb1-4149-acad-b21d6845bf04\") " pod="openshift-dns/dns-default-ntkvz"
Aug 05 17:34:07.290803 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.290766    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b92828f0-7eb1-4149-acad-b21d6845bf04-config-volume\") pod \"dns-default-ntkvz\" (UID: \"b92828f0-7eb1-4149-acad-b21d6845bf04\") " pod="openshift-dns/dns-default-ntkvz"
Aug 05 17:34:07.290840 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.290824    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59kxm\" (UniqueName: \"kubernetes.io/projected/b92828f0-7eb1-4149-acad-b21d6845bf04-kube-api-access-59kxm\") pod \"dns-default-ntkvz\" (UID: \"b92828f0-7eb1-4149-acad-b21d6845bf04\") " pod="openshift-dns/dns-default-ntkvz"
Aug 05 17:34:07.391301 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.391248    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b92828f0-7eb1-4149-acad-b21d6845bf04-config-volume\") pod \"dns-default-ntkvz\" (UID: \"b92828f0-7eb1-4149-acad-b21d6845bf04\") " pod="openshift-dns/dns-default-ntkvz"
Aug 05 17:34:07.391301 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.391289    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59kxm\" (UniqueName: \"kubernetes.io/projected/b92828f0-7eb1-4149-acad-b21d6845bf04-kube-api-access-59kxm\") pod \"dns-default-ntkvz\" (UID: \"b92828f0-7eb1-4149-acad-b21d6845bf04\") " pod="openshift-dns/dns-default-ntkvz"
Aug 05 17:34:07.391446 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.391320    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b92828f0-7eb1-4149-acad-b21d6845bf04-tmp-dir\") pod \"dns-default-ntkvz\" (UID: \"b92828f0-7eb1-4149-acad-b21d6845bf04\") " pod="openshift-dns/dns-default-ntkvz"
Aug 05 17:34:07.391446 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.391361    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cb27b329-ae29-450d-9029-f24d7840c1dc-crio-socket\") pod \"insights-runtime-extractor-s79s8\" (UID: \"cb27b329-ae29-450d-9029-f24d7840c1dc\") " pod="openshift-insights/insights-runtime-extractor-s79s8"
Aug 05 17:34:07.391446 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.391385    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb27b329-ae29-450d-9029-f24d7840c1dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s79s8\" (UID: \"cb27b329-ae29-450d-9029-f24d7840c1dc\") " pod="openshift-insights/insights-runtime-extractor-s79s8"
Aug 05 17:34:07.391446 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.391424    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb27b329-ae29-450d-9029-f24d7840c1dc-data-volume\") pod \"insights-runtime-extractor-s79s8\" (UID: \"cb27b329-ae29-450d-9029-f24d7840c1dc\") " pod="openshift-insights/insights-runtime-extractor-s79s8"
Aug 05 17:34:07.391617 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.391449    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tvvj\" (UniqueName: \"kubernetes.io/projected/cb27b329-ae29-450d-9029-f24d7840c1dc-kube-api-access-6tvvj\") pod \"insights-runtime-extractor-s79s8\" (UID: \"cb27b329-ae29-450d-9029-f24d7840c1dc\") " pod="openshift-insights/insights-runtime-extractor-s79s8"
Aug 05 17:34:07.391617 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.391495    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6e3eb00-c647-435d-aced-22a116ccc2b7-cert\") pod \"ingress-canary-czglh\" (UID: \"d6e3eb00-c647-435d-aced-22a116ccc2b7\") " pod="openshift-ingress-canary/ingress-canary-czglh"
Aug 05 17:34:07.391617 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.391558    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v698j\" (UniqueName: \"kubernetes.io/projected/d6e3eb00-c647-435d-aced-22a116ccc2b7-kube-api-access-v698j\") pod \"ingress-canary-czglh\" (UID: \"d6e3eb00-c647-435d-aced-22a116ccc2b7\") " pod="openshift-ingress-canary/ingress-canary-czglh"
Aug 05 17:34:07.391617 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.391584    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b92828f0-7eb1-4149-acad-b21d6845bf04-metrics-tls\") pod \"dns-default-ntkvz\" (UID: \"b92828f0-7eb1-4149-acad-b21d6845bf04\") " pod="openshift-dns/dns-default-ntkvz"
Aug 05 17:34:07.391617 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.391603    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb27b329-ae29-450d-9029-f24d7840c1dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s79s8\" (UID: \"cb27b329-ae29-450d-9029-f24d7840c1dc\") " pod="openshift-insights/insights-runtime-extractor-s79s8"
Aug 05 17:34:07.391777 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.391643    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b92828f0-7eb1-4149-acad-b21d6845bf04-tmp-dir\") pod \"dns-default-ntkvz\" (UID: \"b92828f0-7eb1-4149-acad-b21d6845bf04\") " pod="openshift-dns/dns-default-ntkvz"
Aug 05 17:34:07.391920 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.391901    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b92828f0-7eb1-4149-acad-b21d6845bf04-config-volume\") pod \"dns-default-ntkvz\" (UID: \"b92828f0-7eb1-4149-acad-b21d6845bf04\") " pod="openshift-dns/dns-default-ntkvz"
Aug 05 17:34:07.395698 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.395681    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b92828f0-7eb1-4149-acad-b21d6845bf04-metrics-tls\") pod \"dns-default-ntkvz\" (UID: \"b92828f0-7eb1-4149-acad-b21d6845bf04\") " pod="openshift-dns/dns-default-ntkvz"
Aug 05 17:34:07.407062 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.407044    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59kxm\" (UniqueName: \"kubernetes.io/projected/b92828f0-7eb1-4149-acad-b21d6845bf04-kube-api-access-59kxm\") pod \"dns-default-ntkvz\" (UID: \"b92828f0-7eb1-4149-acad-b21d6845bf04\") " pod="openshift-dns/dns-default-ntkvz"
Aug 05 17:34:07.420021 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.420004    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ntkvz"
Aug 05 17:34:07.447033 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.447005    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:07.492027 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.491999    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb27b329-ae29-450d-9029-f24d7840c1dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s79s8\" (UID: \"cb27b329-ae29-450d-9029-f24d7840c1dc\") " pod="openshift-insights/insights-runtime-extractor-s79s8"
Aug 05 17:34:07.492135 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.492069    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cb27b329-ae29-450d-9029-f24d7840c1dc-crio-socket\") pod \"insights-runtime-extractor-s79s8\" (UID: \"cb27b329-ae29-450d-9029-f24d7840c1dc\") " pod="openshift-insights/insights-runtime-extractor-s79s8"
Aug 05 17:34:07.492135 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.492090    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb27b329-ae29-450d-9029-f24d7840c1dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s79s8\" (UID: \"cb27b329-ae29-450d-9029-f24d7840c1dc\") " pod="openshift-insights/insights-runtime-extractor-s79s8"
Aug 05 17:34:07.492135 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.492113    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb27b329-ae29-450d-9029-f24d7840c1dc-data-volume\") pod \"insights-runtime-extractor-s79s8\" (UID: \"cb27b329-ae29-450d-9029-f24d7840c1dc\") " pod="openshift-insights/insights-runtime-extractor-s79s8"
Aug 05 17:34:07.492135 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.492128    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tvvj\" (UniqueName: \"kubernetes.io/projected/cb27b329-ae29-450d-9029-f24d7840c1dc-kube-api-access-6tvvj\") pod \"insights-runtime-extractor-s79s8\" (UID: \"cb27b329-ae29-450d-9029-f24d7840c1dc\") " pod="openshift-insights/insights-runtime-extractor-s79s8"
Aug 05 17:34:07.492335 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.492146    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6e3eb00-c647-435d-aced-22a116ccc2b7-cert\") pod \"ingress-canary-czglh\" (UID: \"d6e3eb00-c647-435d-aced-22a116ccc2b7\") " pod="openshift-ingress-canary/ingress-canary-czglh"
Aug 05 17:34:07.492391 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.492363    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cb27b329-ae29-450d-9029-f24d7840c1dc-crio-socket\") pod \"insights-runtime-extractor-s79s8\" (UID: \"cb27b329-ae29-450d-9029-f24d7840c1dc\") " pod="openshift-insights/insights-runtime-extractor-s79s8"
Aug 05 17:34:07.492838 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.492773    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb27b329-ae29-450d-9029-f24d7840c1dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s79s8\" (UID: \"cb27b329-ae29-450d-9029-f24d7840c1dc\") " pod="openshift-insights/insights-runtime-extractor-s79s8"
Aug 05 17:34:07.492838 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.492788    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v698j\" (UniqueName: \"kubernetes.io/projected/d6e3eb00-c647-435d-aced-22a116ccc2b7-kube-api-access-v698j\") pod \"ingress-canary-czglh\" (UID: \"d6e3eb00-c647-435d-aced-22a116ccc2b7\") " pod="openshift-ingress-canary/ingress-canary-czglh"
Aug 05 17:34:07.492838 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.492773    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb27b329-ae29-450d-9029-f24d7840c1dc-data-volume\") pod \"insights-runtime-extractor-s79s8\" (UID: \"cb27b329-ae29-450d-9029-f24d7840c1dc\") " pod="openshift-insights/insights-runtime-extractor-s79s8"
Aug 05 17:34:07.494862 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.494821    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb27b329-ae29-450d-9029-f24d7840c1dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s79s8\" (UID: \"cb27b329-ae29-450d-9029-f24d7840c1dc\") " pod="openshift-insights/insights-runtime-extractor-s79s8"
Aug 05 17:34:07.494950 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.494882    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6e3eb00-c647-435d-aced-22a116ccc2b7-cert\") pod \"ingress-canary-czglh\" (UID: \"d6e3eb00-c647-435d-aced-22a116ccc2b7\") " pod="openshift-ingress-canary/ingress-canary-czglh"
Aug 05 17:34:07.527303 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.527272    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v698j\" (UniqueName: \"kubernetes.io/projected/d6e3eb00-c647-435d-aced-22a116ccc2b7-kube-api-access-v698j\") pod \"ingress-canary-czglh\" (UID: \"d6e3eb00-c647-435d-aced-22a116ccc2b7\") " pod="openshift-ingress-canary/ingress-canary-czglh"
Aug 05 17:34:07.528384 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.528366    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tvvj\" (UniqueName: \"kubernetes.io/projected/cb27b329-ae29-450d-9029-f24d7840c1dc-kube-api-access-6tvvj\") pod \"insights-runtime-extractor-s79s8\" (UID: \"cb27b329-ae29-450d-9029-f24d7840c1dc\") " pod="openshift-insights/insights-runtime-extractor-s79s8"
Aug 05 17:34:07.529226 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.529208    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-czglh"
Aug 05 17:34:07.553904 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.553883    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s79s8"
Aug 05 17:34:07.595419 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.595392    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ntkvz"]
Aug 05 17:34:07.595793 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:34:07.595773    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb92828f0_7eb1_4149_acad_b21d6845bf04.slice/crio-6535324a55acf4f78a635162df00db560a66fa1f0aaea83a174d32aa9a2b0419 WatchSource:0}: Error finding container 6535324a55acf4f78a635162df00db560a66fa1f0aaea83a174d32aa9a2b0419: Status 404 returned error can't find the container with id 6535324a55acf4f78a635162df00db560a66fa1f0aaea83a174d32aa9a2b0419
Aug 05 17:34:07.646901 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.646823    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:07.679633 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.679607    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-czglh"]
Aug 05 17:34:07.682675 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:34:07.682654    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6e3eb00_c647_435d_aced_22a116ccc2b7.slice/crio-94302f0df22a93d7d9b750d2fd13adc97ded814248c3f3d39739f1beccf709a5 WatchSource:0}: Error finding container 94302f0df22a93d7d9b750d2fd13adc97ded814248c3f3d39739f1beccf709a5: Status 404 returned error can't find the container with id 94302f0df22a93d7d9b750d2fd13adc97ded814248c3f3d39739f1beccf709a5
Aug 05 17:34:07.710301 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:07.710280    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s79s8"]
Aug 05 17:34:07.712731 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:34:07.712712    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb27b329_ae29_450d_9029_f24d7840c1dc.slice/crio-8dab748e4e7fe4a4551be6f1700d875f2d2f2b644de67b9aed2fe2159c18a565 WatchSource:0}: Error finding container 8dab748e4e7fe4a4551be6f1700d875f2d2f2b644de67b9aed2fe2159c18a565: Status 404 returned error can't find the container with id 8dab748e4e7fe4a4551be6f1700d875f2d2f2b644de67b9aed2fe2159c18a565
Aug 05 17:34:08.144695 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:08.144665    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:34:08.144875 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:08.144665    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:34:08.144875 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:08.144663    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:34:08.147820 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:08.147799    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 05 17:34:08.148262 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:08.148215    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 05 17:34:08.149273 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:08.149251    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bbv86\""
Aug 05 17:34:08.149273 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:08.149271    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dt589\""
Aug 05 17:34:08.149543 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:08.149527    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 05 17:34:08.149604 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:08.149574    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 05 17:34:08.477987 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:08.477834    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s79s8" event={"ID":"cb27b329-ae29-450d-9029-f24d7840c1dc","Type":"ContainerStarted","Data":"913c3aa21123541fda1b05501a3fed0f62123b9477a909bf2e44c41c5878a3a4"}
Aug 05 17:34:08.477987 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:08.477908    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s79s8" event={"ID":"cb27b329-ae29-450d-9029-f24d7840c1dc","Type":"ContainerStarted","Data":"8dab748e4e7fe4a4551be6f1700d875f2d2f2b644de67b9aed2fe2159c18a565"}
Aug 05 17:34:08.480998 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:08.479555    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-czglh" event={"ID":"d6e3eb00-c647-435d-aced-22a116ccc2b7","Type":"ContainerStarted","Data":"94302f0df22a93d7d9b750d2fd13adc97ded814248c3f3d39739f1beccf709a5"}
Aug 05 17:34:08.480998 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:08.480956    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ntkvz" event={"ID":"b92828f0-7eb1-4149-acad-b21d6845bf04","Type":"ContainerStarted","Data":"6535324a55acf4f78a635162df00db560a66fa1f0aaea83a174d32aa9a2b0419"}
Aug 05 17:34:08.906705 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:08.906673    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:09.005758 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:09.005728    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:09.024027 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:09.023999    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-q5dht"]
Aug 05 17:34:09.026347 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:09.026319    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-q5dht"
Aug 05 17:34:09.029363 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:09.028847    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Aug 05 17:34:09.029363 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:09.029137    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-ltqkl\""
Aug 05 17:34:09.039371 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:09.039292    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-q5dht"]
Aug 05 17:34:09.075818 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:09.075788    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:09.106371 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:09.106336    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/22a6cba0-5553-42d8-b6c6-83a5c261eb34-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-q5dht\" (UID: \"22a6cba0-5553-42d8-b6c6-83a5c261eb34\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-q5dht"
Aug 05 17:34:09.184637 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:09.184549    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:09.206764 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:09.206731    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/22a6cba0-5553-42d8-b6c6-83a5c261eb34-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-q5dht\" (UID: \"22a6cba0-5553-42d8-b6c6-83a5c261eb34\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-q5dht"
Aug 05 17:34:09.209861 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:09.209834    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/22a6cba0-5553-42d8-b6c6-83a5c261eb34-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-q5dht\" (UID: \"22a6cba0-5553-42d8-b6c6-83a5c261eb34\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-q5dht"
Aug 05 17:34:09.338271 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:09.338244    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-q5dht"
Aug 05 17:34:09.493333 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:09.493296    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s79s8" event={"ID":"cb27b329-ae29-450d-9029-f24d7840c1dc","Type":"ContainerStarted","Data":"01efb043ecb1009026643b5bf98ea2ba7781f0583d3854ebbaad7226d9985d0e"}
Aug 05 17:34:09.496723 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:09.496683    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ntkvz" event={"ID":"b92828f0-7eb1-4149-acad-b21d6845bf04","Type":"ContainerStarted","Data":"7056b9d922ecceb095c6316038b107c0e7aa8d0929572063f2bfc739169361da"}
Aug 05 17:34:09.503658 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:09.503603    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-q5dht"]
Aug 05 17:34:09.511479 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:34:09.511454    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22a6cba0_5553_42d8_b6c6_83a5c261eb34.slice/crio-197e82ff67492d4c6cb88b785363c74abf2574623b62e9a197ab35ed5d1dec0b WatchSource:0}: Error finding container 197e82ff67492d4c6cb88b785363c74abf2574623b62e9a197ab35ed5d1dec0b: Status 404 returned error can't find the container with id 197e82ff67492d4c6cb88b785363c74abf2574623b62e9a197ab35ed5d1dec0b
Aug 05 17:34:10.500990 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:10.500940    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-q5dht" event={"ID":"22a6cba0-5553-42d8-b6c6-83a5c261eb34","Type":"ContainerStarted","Data":"197e82ff67492d4c6cb88b785363c74abf2574623b62e9a197ab35ed5d1dec0b"}
Aug 05 17:34:10.502892 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:10.502867    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ntkvz" event={"ID":"b92828f0-7eb1-4149-acad-b21d6845bf04","Type":"ContainerStarted","Data":"86232eb450fd4fb8745db593a71279daae44648d7ff5257a91734df6d612ad1f"}
Aug 05 17:34:10.503016 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:10.502990    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ntkvz"
Aug 05 17:34:10.529411 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:10.529362    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ntkvz" podStartSLOduration=1.819141366 podStartE2EDuration="3.529348165s" totalImagesPullingTime="1.710206799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:34:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:10.528741845 +0000 UTC m=+57.881416437" watchObservedRunningTime="2026-08-05 17:34:10.529348165 +0000 UTC m=+57.882022759"
Aug 05 17:34:11.508381 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:11.508345    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-q5dht" event={"ID":"22a6cba0-5553-42d8-b6c6-83a5c261eb34","Type":"ContainerStarted","Data":"df1c72302729f4be87efda37e841c29d3b64157c9e0b9ea06988bf40e83f2548"}
Aug 05 17:34:11.508773 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:11.508403    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-q5dht"
Aug 05 17:34:11.510706 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:11.510680    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s79s8" event={"ID":"cb27b329-ae29-450d-9029-f24d7840c1dc","Type":"ContainerStarted","Data":"61008cc4780ddc8069ddcfafc12749552a7662c400434845c979d04782b35094"}
Aug 05 17:34:11.512234 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:11.512211    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-czglh" event={"ID":"d6e3eb00-c647-435d-aced-22a116ccc2b7","Type":"ContainerStarted","Data":"73e2890d4538c16958ae71b64fd8b8529f92d1246b43f7954943d67eefbd990d"}
Aug 05 17:34:11.513730 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:11.513712    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-q5dht"
Aug 05 17:34:11.524217 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:11.524151    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-q5dht" podStartSLOduration=0.663284818 podStartE2EDuration="2.524139422s" totalImagesPullingTime="1.860854604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:11.523406759 +0000 UTC m=+58.876081353" watchObservedRunningTime="2026-08-05 17:34:11.524139422 +0000 UTC m=+58.876814016"
Aug 05 17:34:11.558331 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:11.558288    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-s79s8" podStartSLOduration=1.710539481 podStartE2EDuration="4.558172645s" totalImagesPullingTime="2.847633164s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:34:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:11.557328249 +0000 UTC m=+58.910002846" watchObservedRunningTime="2026-08-05 17:34:11.558172645 +0000 UTC m=+58.910847216"
Aug 05 17:34:11.573057 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:11.573018    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-czglh" podStartSLOduration=1.5614186860000001 podStartE2EDuration="4.572998364s" totalImagesPullingTime="3.011579678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:34:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:11.572919728 +0000 UTC m=+58.925594322" watchObservedRunningTime="2026-08-05 17:34:11.572998364 +0000 UTC m=+58.925672958"
Aug 05 17:34:12.051463 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.051423    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:12.101435 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.101406    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-fdxhn"]
Aug 05 17:34:12.103313 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.103299    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-fdxhn"
Aug 05 17:34:12.108096 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.108071    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Aug 05 17:34:12.108278 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.108147    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Aug 05 17:34:12.108278 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.108268    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Aug 05 17:34:12.108416 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.108150    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Aug 05 17:34:12.108416 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.108191    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Aug 05 17:34:12.108416 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.108214    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-9qxp5\""
Aug 05 17:34:12.116914 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.116898    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-fdxhn"]
Aug 05 17:34:12.134290 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.134266    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-fdxhn\" (UID: \"fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-fdxhn"
Aug 05 17:34:12.134400 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.134320    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7qhk\" (UniqueName: \"kubernetes.io/projected/fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b-kube-api-access-q7qhk\") pod \"prometheus-operator-758d7cb874-fdxhn\" (UID: \"fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-fdxhn"
Aug 05 17:34:12.134400 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.134383    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-fdxhn\" (UID: \"fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-fdxhn"
Aug 05 17:34:12.134513 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.134405    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-fdxhn\" (UID: \"fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-fdxhn"
Aug 05 17:34:12.139798 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.139767    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:12.235575 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.235544    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-fdxhn\" (UID: \"fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-fdxhn"
Aug 05 17:34:12.235738 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.235579    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-fdxhn\" (UID: \"fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-fdxhn"
Aug 05 17:34:12.235803 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.235738    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-fdxhn\" (UID: \"fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-fdxhn"
Aug 05 17:34:12.235803 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.235786    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7qhk\" (UniqueName: \"kubernetes.io/projected/fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b-kube-api-access-q7qhk\") pod \"prometheus-operator-758d7cb874-fdxhn\" (UID: \"fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-fdxhn"
Aug 05 17:34:12.236231 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.236205    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-fdxhn\" (UID: \"fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-fdxhn"
Aug 05 17:34:12.238238 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.238219    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-fdxhn\" (UID: \"fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-fdxhn"
Aug 05 17:34:12.238321 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.238226    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-fdxhn\" (UID: \"fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-fdxhn"
Aug 05 17:34:12.243675 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.243649    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:12.244213 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.244195    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7qhk\" (UniqueName: \"kubernetes.io/projected/fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b-kube-api-access-q7qhk\") pod \"prometheus-operator-758d7cb874-fdxhn\" (UID: \"fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-fdxhn"
Aug 05 17:34:12.411805 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.411768    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-fdxhn"
Aug 05 17:34:12.536215 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:12.536170    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-fdxhn"]
Aug 05 17:34:12.539371 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:34:12.539350    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa95a02b_aa5c_4a7b_8e37_1a18c74ad92b.slice/crio-d311796147c8412f47c86cc7a6e6b83dfa78ca79791d77f4bda9783c9fff708f WatchSource:0}: Error finding container d311796147c8412f47c86cc7a6e6b83dfa78ca79791d77f4bda9783c9fff708f: Status 404 returned error can't find the container with id d311796147c8412f47c86cc7a6e6b83dfa78ca79791d77f4bda9783c9fff708f
Aug 05 17:34:13.519805 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:13.519765    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-fdxhn" event={"ID":"fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b","Type":"ContainerStarted","Data":"d311796147c8412f47c86cc7a6e6b83dfa78ca79791d77f4bda9783c9fff708f"}
Aug 05 17:34:13.728621 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:13.728589    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:13.794279 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:13.794206    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:14.166813 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.166778    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f86985667-9gbwx"]
Aug 05 17:34:14.177403 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.177371    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.180527 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.180507    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Aug 05 17:34:14.180797 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.180784    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-f7vqk\""
Aug 05 17:34:14.181600 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.181579    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Aug 05 17:34:14.181677 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.181628    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Aug 05 17:34:14.182838 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.182815    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Aug 05 17:34:14.182935 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.182845    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Aug 05 17:34:14.183103 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.183090    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Aug 05 17:34:14.183355 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.183343    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Aug 05 17:34:14.187227 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.187206    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f86985667-9gbwx"]
Aug 05 17:34:14.251684 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.251657    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f03449d4-b5a5-4908-8119-ee0ec48e10a3-oauth-serving-cert\") pod \"console-7f86985667-9gbwx\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") " pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.251828 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.251742    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f03449d4-b5a5-4908-8119-ee0ec48e10a3-console-serving-cert\") pod \"console-7f86985667-9gbwx\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") " pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.251828 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.251762    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f03449d4-b5a5-4908-8119-ee0ec48e10a3-console-oauth-config\") pod \"console-7f86985667-9gbwx\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") " pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.251828 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.251777    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f03449d4-b5a5-4908-8119-ee0ec48e10a3-service-ca\") pod \"console-7f86985667-9gbwx\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") " pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.251828 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.251795    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qts7r\" (UniqueName: \"kubernetes.io/projected/f03449d4-b5a5-4908-8119-ee0ec48e10a3-kube-api-access-qts7r\") pod \"console-7f86985667-9gbwx\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") " pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.251828 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.251814    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f03449d4-b5a5-4908-8119-ee0ec48e10a3-console-config\") pod \"console-7f86985667-9gbwx\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") " pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.353098 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.353062    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f03449d4-b5a5-4908-8119-ee0ec48e10a3-oauth-serving-cert\") pod \"console-7f86985667-9gbwx\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") " pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.353297 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.353195    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f03449d4-b5a5-4908-8119-ee0ec48e10a3-console-serving-cert\") pod \"console-7f86985667-9gbwx\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") " pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.353297 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.353224    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f03449d4-b5a5-4908-8119-ee0ec48e10a3-console-oauth-config\") pod \"console-7f86985667-9gbwx\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") " pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.353297 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.353249    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f03449d4-b5a5-4908-8119-ee0ec48e10a3-service-ca\") pod \"console-7f86985667-9gbwx\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") " pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.353297 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.353274    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qts7r\" (UniqueName: \"kubernetes.io/projected/f03449d4-b5a5-4908-8119-ee0ec48e10a3-kube-api-access-qts7r\") pod \"console-7f86985667-9gbwx\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") " pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.353441 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.353304    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f03449d4-b5a5-4908-8119-ee0ec48e10a3-console-config\") pod \"console-7f86985667-9gbwx\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") " pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.353853 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.353821    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f03449d4-b5a5-4908-8119-ee0ec48e10a3-oauth-serving-cert\") pod \"console-7f86985667-9gbwx\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") " pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.353991 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.353966    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f03449d4-b5a5-4908-8119-ee0ec48e10a3-service-ca\") pod \"console-7f86985667-9gbwx\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") " pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.353991 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.353988    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f03449d4-b5a5-4908-8119-ee0ec48e10a3-console-config\") pod \"console-7f86985667-9gbwx\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") " pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.356258 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.356238    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f03449d4-b5a5-4908-8119-ee0ec48e10a3-console-oauth-config\") pod \"console-7f86985667-9gbwx\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") " pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.356331 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.356288    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f03449d4-b5a5-4908-8119-ee0ec48e10a3-console-serving-cert\") pod \"console-7f86985667-9gbwx\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") " pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.362837 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.362817    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qts7r\" (UniqueName: \"kubernetes.io/projected/f03449d4-b5a5-4908-8119-ee0ec48e10a3-kube-api-access-qts7r\") pod \"console-7f86985667-9gbwx\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") " pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.434441 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.434368    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xpsg4"
Aug 05 17:34:14.487102 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.487068    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:14.570196 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.570158    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:14.629166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.629146    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f86985667-9gbwx"]
Aug 05 17:34:14.632312 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:34:14.632287    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf03449d4_b5a5_4908_8119_ee0ec48e10a3.slice/crio-c1154c534a8b7e6814fdb0d5092a318ab4169dff5108d9261d6590ffc2c8f9fd WatchSource:0}: Error finding container c1154c534a8b7e6814fdb0d5092a318ab4169dff5108d9261d6590ffc2c8f9fd: Status 404 returned error can't find the container with id c1154c534a8b7e6814fdb0d5092a318ab4169dff5108d9261d6590ffc2c8f9fd
Aug 05 17:34:14.634840 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.634818    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:14.781200 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:14.781143    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:15.102132 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:15.102091    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:15.199970 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:15.199933    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:15.528328 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:15.528241    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-fdxhn" event={"ID":"fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b","Type":"ContainerStarted","Data":"47f7e1af3d142478ffc6f21dc36c73efa8b6ce568f070a81e82b494968bc9ada"}
Aug 05 17:34:15.528328 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:15.528300    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-fdxhn" event={"ID":"fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b","Type":"ContainerStarted","Data":"111ea0fa60fa643e5cd320d3baa8b9adc0bf4ec2f7aeca4aad7f37d557cbc3eb"}
Aug 05 17:34:15.529744 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:15.529712    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f86985667-9gbwx" event={"ID":"f03449d4-b5a5-4908-8119-ee0ec48e10a3","Type":"ContainerStarted","Data":"c1154c534a8b7e6814fdb0d5092a318ab4169dff5108d9261d6590ffc2c8f9fd"}
Aug 05 17:34:15.548697 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:15.548400    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-758d7cb874-fdxhn" podStartSLOduration=1.624361275 podStartE2EDuration="3.548380811s" totalImagesPullingTime="1.924019536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:15.547513429 +0000 UTC m=+62.900188024" watchObservedRunningTime="2026-08-05 17:34:15.548380811 +0000 UTC m=+62.901055406"
Aug 05 17:34:15.933080 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:15.933041    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:16.014286 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:16.014253    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:17.222446 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.222420    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:17.458589 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.458546    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:17.498523 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.498447    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz"]
Aug 05 17:34:17.504667 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.504642    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"]
Aug 05 17:34:17.504816 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.504788    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz"
Aug 05 17:34:17.506787 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.506762    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.509041 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.509015    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Aug 05 17:34:17.510160 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.510129    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Aug 05 17:34:17.510979 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.510961    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Aug 05 17:34:17.511358 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.511332    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-w69vr"]
Aug 05 17:34:17.511467 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.511391    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-gbxcv\""
Aug 05 17:34:17.511467 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.511404    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Aug 05 17:34:17.511467 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.511430    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-5rcvt\""
Aug 05 17:34:17.511661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.511638    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Aug 05 17:34:17.513785 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.513765    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.516622 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.516257    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Aug 05 17:34:17.516622 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.516445    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Aug 05 17:34:17.516622 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.516496    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Aug 05 17:34:17.516763 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.516646    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"]
Aug 05 17:34:17.516763 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.516691    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-p8psp\""
Aug 05 17:34:17.519706 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.519686    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz"]
Aug 05 17:34:17.580654 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.580620    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8721c842-6692-47bd-a37a-3c1d48edc59a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-855bm\" (UID: \"8721c842-6692-47bd-a37a-3c1d48edc59a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.580829 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.580671    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2c558452-bf1d-44ba-a169-a78f38dc4bb3-node-exporter-tls\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.580829 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.580706    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ee151b3-b9ea-499f-afd4-816e849c1c5f-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-vmbsz\" (UID: \"9ee151b3-b9ea-499f-afd4-816e849c1c5f\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz"
Aug 05 17:34:17.580829 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.580730    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c558452-bf1d-44ba-a169-a78f38dc4bb3-sys\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.580829 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.580751    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2c558452-bf1d-44ba-a169-a78f38dc4bb3-root\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.581021 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.580845    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8721c842-6692-47bd-a37a-3c1d48edc59a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-855bm\" (UID: \"8721c842-6692-47bd-a37a-3c1d48edc59a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.581021 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.580890    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4scsv\" (UniqueName: \"kubernetes.io/projected/9ee151b3-b9ea-499f-afd4-816e849c1c5f-kube-api-access-4scsv\") pod \"openshift-state-metrics-6bb4d7b4dd-vmbsz\" (UID: \"9ee151b3-b9ea-499f-afd4-816e849c1c5f\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz"
Aug 05 17:34:17.581021 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.580936    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb8dd\" (UniqueName: \"kubernetes.io/projected/8721c842-6692-47bd-a37a-3c1d48edc59a-kube-api-access-pb8dd\") pod \"kube-state-metrics-6bd9664c4-855bm\" (UID: \"8721c842-6692-47bd-a37a-3c1d48edc59a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.581021 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.580969    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c558452-bf1d-44ba-a169-a78f38dc4bb3-metrics-client-ca\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.581225 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.581025    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8721c842-6692-47bd-a37a-3c1d48edc59a-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-855bm\" (UID: \"8721c842-6692-47bd-a37a-3c1d48edc59a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.581225 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.581052    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ee151b3-b9ea-499f-afd4-816e849c1c5f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-vmbsz\" (UID: \"9ee151b3-b9ea-499f-afd4-816e849c1c5f\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz"
Aug 05 17:34:17.581225 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.581080    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c558452-bf1d-44ba-a169-a78f38dc4bb3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.581225 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.581112    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2c558452-bf1d-44ba-a169-a78f38dc4bb3-node-exporter-wtmp\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.581225 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.581142    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2c558452-bf1d-44ba-a169-a78f38dc4bb3-node-exporter-textfile\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.581225 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.581172    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2c558452-bf1d-44ba-a169-a78f38dc4bb3-node-exporter-accelerators-collector-config\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.581225 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.581214    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8721c842-6692-47bd-a37a-3c1d48edc59a-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-855bm\" (UID: \"8721c842-6692-47bd-a37a-3c1d48edc59a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.581487 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.581281    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbfcf\" (UniqueName: \"kubernetes.io/projected/2c558452-bf1d-44ba-a169-a78f38dc4bb3-kube-api-access-lbfcf\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.581487 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.581307    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8721c842-6692-47bd-a37a-3c1d48edc59a-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-855bm\" (UID: \"8721c842-6692-47bd-a37a-3c1d48edc59a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.581487 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.581334    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ee151b3-b9ea-499f-afd4-816e849c1c5f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-vmbsz\" (UID: \"9ee151b3-b9ea-499f-afd4-816e849c1c5f\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz"
Aug 05 17:34:17.583155 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.583095    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:17.682350 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682315    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ee151b3-b9ea-499f-afd4-816e849c1c5f-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-vmbsz\" (UID: \"9ee151b3-b9ea-499f-afd4-816e849c1c5f\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz"
Aug 05 17:34:17.682526 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682363    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c558452-bf1d-44ba-a169-a78f38dc4bb3-sys\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.682526 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682385    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2c558452-bf1d-44ba-a169-a78f38dc4bb3-root\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.682526 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682427    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8721c842-6692-47bd-a37a-3c1d48edc59a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-855bm\" (UID: \"8721c842-6692-47bd-a37a-3c1d48edc59a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.682526 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682448    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c558452-bf1d-44ba-a169-a78f38dc4bb3-sys\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.682526 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682461    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4scsv\" (UniqueName: \"kubernetes.io/projected/9ee151b3-b9ea-499f-afd4-816e849c1c5f-kube-api-access-4scsv\") pod \"openshift-state-metrics-6bb4d7b4dd-vmbsz\" (UID: \"9ee151b3-b9ea-499f-afd4-816e849c1c5f\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz"
Aug 05 17:34:17.682526 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682522    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb8dd\" (UniqueName: \"kubernetes.io/projected/8721c842-6692-47bd-a37a-3c1d48edc59a-kube-api-access-pb8dd\") pod \"kube-state-metrics-6bd9664c4-855bm\" (UID: \"8721c842-6692-47bd-a37a-3c1d48edc59a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.682897 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682550    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c558452-bf1d-44ba-a169-a78f38dc4bb3-metrics-client-ca\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.682897 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682605    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8721c842-6692-47bd-a37a-3c1d48edc59a-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-855bm\" (UID: \"8721c842-6692-47bd-a37a-3c1d48edc59a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.682897 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682631    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ee151b3-b9ea-499f-afd4-816e849c1c5f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-vmbsz\" (UID: \"9ee151b3-b9ea-499f-afd4-816e849c1c5f\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz"
Aug 05 17:34:17.682897 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682664    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c558452-bf1d-44ba-a169-a78f38dc4bb3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.682897 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682695    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2c558452-bf1d-44ba-a169-a78f38dc4bb3-node-exporter-wtmp\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.682897 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682726    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2c558452-bf1d-44ba-a169-a78f38dc4bb3-node-exporter-textfile\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.682897 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682757    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2c558452-bf1d-44ba-a169-a78f38dc4bb3-node-exporter-accelerators-collector-config\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.682897 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682785    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8721c842-6692-47bd-a37a-3c1d48edc59a-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-855bm\" (UID: \"8721c842-6692-47bd-a37a-3c1d48edc59a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.682897 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682798    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2c558452-bf1d-44ba-a169-a78f38dc4bb3-root\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.682897 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682843    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbfcf\" (UniqueName: \"kubernetes.io/projected/2c558452-bf1d-44ba-a169-a78f38dc4bb3-kube-api-access-lbfcf\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.682897 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682872    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8721c842-6692-47bd-a37a-3c1d48edc59a-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-855bm\" (UID: \"8721c842-6692-47bd-a37a-3c1d48edc59a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.682897 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682898    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ee151b3-b9ea-499f-afd4-816e849c1c5f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-vmbsz\" (UID: \"9ee151b3-b9ea-499f-afd4-816e849c1c5f\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz"
Aug 05 17:34:17.683480 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682933    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8721c842-6692-47bd-a37a-3c1d48edc59a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-855bm\" (UID: \"8721c842-6692-47bd-a37a-3c1d48edc59a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.683480 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.682964    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2c558452-bf1d-44ba-a169-a78f38dc4bb3-node-exporter-tls\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.683480 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.683070    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ee151b3-b9ea-499f-afd4-816e849c1c5f-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-vmbsz\" (UID: \"9ee151b3-b9ea-499f-afd4-816e849c1c5f\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz"
Aug 05 17:34:17.683480 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:34:17.683105    2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Aug 05 17:34:17.683480 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:34:17.683191    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c558452-bf1d-44ba-a169-a78f38dc4bb3-node-exporter-tls podName:2c558452-bf1d-44ba-a169-a78f38dc4bb3 nodeName:}" failed. No retries permitted until 2026-08-05 17:34:18.183154118 +0000 UTC m=+65.535828696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2c558452-bf1d-44ba-a169-a78f38dc4bb3-node-exporter-tls") pod "node-exporter-w69vr" (UID: "2c558452-bf1d-44ba-a169-a78f38dc4bb3") : secret "node-exporter-tls" not found
Aug 05 17:34:17.683480 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.683383    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2c558452-bf1d-44ba-a169-a78f38dc4bb3-node-exporter-textfile\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.683760 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.683486    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8721c842-6692-47bd-a37a-3c1d48edc59a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-855bm\" (UID: \"8721c842-6692-47bd-a37a-3c1d48edc59a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.683822 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.683764    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8721c842-6692-47bd-a37a-3c1d48edc59a-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-855bm\" (UID: \"8721c842-6692-47bd-a37a-3c1d48edc59a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.683912 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.683893    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2c558452-bf1d-44ba-a169-a78f38dc4bb3-node-exporter-accelerators-collector-config\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.684009 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.683917    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c558452-bf1d-44ba-a169-a78f38dc4bb3-metrics-client-ca\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.684732 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.684505    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8721c842-6692-47bd-a37a-3c1d48edc59a-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-855bm\" (UID: \"8721c842-6692-47bd-a37a-3c1d48edc59a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.684732 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.684654    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2c558452-bf1d-44ba-a169-a78f38dc4bb3-node-exporter-wtmp\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.691161 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.690735    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ee151b3-b9ea-499f-afd4-816e849c1c5f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-vmbsz\" (UID: \"9ee151b3-b9ea-499f-afd4-816e849c1c5f\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz"
Aug 05 17:34:17.691161 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.690749    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ee151b3-b9ea-499f-afd4-816e849c1c5f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-vmbsz\" (UID: \"9ee151b3-b9ea-499f-afd4-816e849c1c5f\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz"
Aug 05 17:34:17.691161 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.690760    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8721c842-6692-47bd-a37a-3c1d48edc59a-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-855bm\" (UID: \"8721c842-6692-47bd-a37a-3c1d48edc59a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.691161 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.690944    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8721c842-6692-47bd-a37a-3c1d48edc59a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-855bm\" (UID: \"8721c842-6692-47bd-a37a-3c1d48edc59a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.691161 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.690983    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c558452-bf1d-44ba-a169-a78f38dc4bb3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.693621 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.693561    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4scsv\" (UniqueName: \"kubernetes.io/projected/9ee151b3-b9ea-499f-afd4-816e849c1c5f-kube-api-access-4scsv\") pod \"openshift-state-metrics-6bb4d7b4dd-vmbsz\" (UID: \"9ee151b3-b9ea-499f-afd4-816e849c1c5f\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz"
Aug 05 17:34:17.697074 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.697052    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb8dd\" (UniqueName: \"kubernetes.io/projected/8721c842-6692-47bd-a37a-3c1d48edc59a-kube-api-access-pb8dd\") pod \"kube-state-metrics-6bd9664c4-855bm\" (UID: \"8721c842-6692-47bd-a37a-3c1d48edc59a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.700552 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.700529    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbfcf\" (UniqueName: \"kubernetes.io/projected/2c558452-bf1d-44ba-a169-a78f38dc4bb3-kube-api-access-lbfcf\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:17.735901 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.733160    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:17.818817 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.818794    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz"
Aug 05 17:34:17.827261 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.827239    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"
Aug 05 17:34:17.934142 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.934082    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:17.978843 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.978809    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz"]
Aug 05 17:34:17.981768 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:34:17.981741    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ee151b3_b9ea_499f_afd4_816e849c1c5f.slice/crio-13fab58cf49197b4e2f8c61ea03b8ecea96b1a4efd6e0d9df7586bf7df186a6b WatchSource:0}: Error finding container 13fab58cf49197b4e2f8c61ea03b8ecea96b1a4efd6e0d9df7586bf7df186a6b: Status 404 returned error can't find the container with id 13fab58cf49197b4e2f8c61ea03b8ecea96b1a4efd6e0d9df7586bf7df186a6b
Aug 05 17:34:17.984894 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:17.984871    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-855bm"]
Aug 05 17:34:17.987679 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:34:17.987656    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8721c842_6692_47bd_a37a_3c1d48edc59a.slice/crio-6e84c02790cdae49b7f1c2a81495cf3176f6015b4876532ccd57b5ac1847a760 WatchSource:0}: Error finding container 6e84c02790cdae49b7f1c2a81495cf3176f6015b4876532ccd57b5ac1847a760: Status 404 returned error can't find the container with id 6e84c02790cdae49b7f1c2a81495cf3176f6015b4876532ccd57b5ac1847a760
Aug 05 17:34:18.061597 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.061567    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:18.188279 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.188249    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2c558452-bf1d-44ba-a169-a78f38dc4bb3-node-exporter-tls\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:18.190697 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.190677    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2c558452-bf1d-44ba-a169-a78f38dc4bb3-node-exporter-tls\") pod \"node-exporter-w69vr\" (UID: \"2c558452-bf1d-44ba-a169-a78f38dc4bb3\") " pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:18.434432 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.434401    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w69vr"
Aug 05 17:34:18.444114 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:34:18.444081    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c558452_bf1d_44ba_a169_a78f38dc4bb3.slice/crio-46b6eee300433d2feb439fbbfb4a9432ffb19fb4b460c5219b2ab5ce31b11dc4 WatchSource:0}: Error finding container 46b6eee300433d2feb439fbbfb4a9432ffb19fb4b460c5219b2ab5ce31b11dc4: Status 404 returned error can't find the container with id 46b6eee300433d2feb439fbbfb4a9432ffb19fb4b460c5219b2ab5ce31b11dc4
Aug 05 17:34:18.506138 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.506109    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:18.541249 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.541210    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm" event={"ID":"8721c842-6692-47bd-a37a-3c1d48edc59a","Type":"ContainerStarted","Data":"6e84c02790cdae49b7f1c2a81495cf3176f6015b4876532ccd57b5ac1847a760"}
Aug 05 17:34:18.543150 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.543116    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz" event={"ID":"9ee151b3-b9ea-499f-afd4-816e849c1c5f","Type":"ContainerStarted","Data":"a84db608044426c9226dcf52128f58f7b018f4914b96fecb09f6e1269781d019"}
Aug 05 17:34:18.543294 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.543204    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz" event={"ID":"9ee151b3-b9ea-499f-afd4-816e849c1c5f","Type":"ContainerStarted","Data":"f16dd6d92b85dea4fb6b8e9e401af6427bbb8e69060e6e9e8db65c9da8a12747"}
Aug 05 17:34:18.543294 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.543233    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz" event={"ID":"9ee151b3-b9ea-499f-afd4-816e849c1c5f","Type":"ContainerStarted","Data":"13fab58cf49197b4e2f8c61ea03b8ecea96b1a4efd6e0d9df7586bf7df186a6b"}
Aug 05 17:34:18.544589 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.544564    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f86985667-9gbwx" event={"ID":"f03449d4-b5a5-4908-8119-ee0ec48e10a3","Type":"ContainerStarted","Data":"a07227869bd4dcf076f5988c9b123dce8b5796ff31b2a9386462b67a776dc684"}
Aug 05 17:34:18.545870 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.545845    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w69vr" event={"ID":"2c558452-bf1d-44ba-a169-a78f38dc4bb3","Type":"ContainerStarted","Data":"46b6eee300433d2feb439fbbfb4a9432ffb19fb4b460c5219b2ab5ce31b11dc4"}
Aug 05 17:34:18.567196 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.567134    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f86985667-9gbwx" podStartSLOduration=1.414280757 podStartE2EDuration="4.567116276s" totalImagesPullingTime="3.152835519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:34:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:18.566262862 +0000 UTC m=+65.918937457" watchObservedRunningTime="2026-08-05 17:34:18.567116276 +0000 UTC m=+65.919790870"
Aug 05 17:34:18.600607 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.600562    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:18.618089 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.618061    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 05 17:34:18.621453 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.621433    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.625170 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.624037    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Aug 05 17:34:18.625170 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.624130    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Aug 05 17:34:18.625170 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.624393    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Aug 05 17:34:18.625170 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.624433    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Aug 05 17:34:18.625170 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.624663    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Aug 05 17:34:18.625170 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.624732    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jcwdk\""
Aug 05 17:34:18.625170 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.624665    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Aug 05 17:34:18.625170 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.624996    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Aug 05 17:34:18.625170 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.625018    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Aug 05 17:34:18.625569 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.625240    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Aug 05 17:34:18.641700 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.641670    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 05 17:34:18.692220 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.692120    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmgq4\" (UniqueName: \"kubernetes.io/projected/39dfc855-2987-4dab-b0ca-8225ad97d1d9-kube-api-access-hmgq4\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.692220 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.692196    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.692424 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.692266    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39dfc855-2987-4dab-b0ca-8225ad97d1d9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.692424 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.692333    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.692424 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.692379    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39dfc855-2987-4dab-b0ca-8225ad97d1d9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.692569 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.692446    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.692569 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.692472    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39dfc855-2987-4dab-b0ca-8225ad97d1d9-config-out\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.692650 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.692573    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.692650 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.692612    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/39dfc855-2987-4dab-b0ca-8225ad97d1d9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.692733 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.692661    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-web-config\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.692733 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.692711    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.692811 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.692769    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-config-volume\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.692811 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.692788    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39dfc855-2987-4dab-b0ca-8225ad97d1d9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.793771 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.793742    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.793951 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.793787    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/39dfc855-2987-4dab-b0ca-8225ad97d1d9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.794023 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.793959    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-web-config\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.794023 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.793998    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.794130 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.794063    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-config-volume\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.794130 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.794085    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39dfc855-2987-4dab-b0ca-8225ad97d1d9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.794130 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.794117    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmgq4\" (UniqueName: \"kubernetes.io/projected/39dfc855-2987-4dab-b0ca-8225ad97d1d9-kube-api-access-hmgq4\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.794377 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.794149    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.794377 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.794173    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39dfc855-2987-4dab-b0ca-8225ad97d1d9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.794377 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.794223    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.794377 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.794253    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39dfc855-2987-4dab-b0ca-8225ad97d1d9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.794377 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.794311    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.794377 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.794342    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39dfc855-2987-4dab-b0ca-8225ad97d1d9-config-out\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.794779 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.794171    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/39dfc855-2987-4dab-b0ca-8225ad97d1d9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.796202 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.796137    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39dfc855-2987-4dab-b0ca-8225ad97d1d9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.797402 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.797368    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39dfc855-2987-4dab-b0ca-8225ad97d1d9-config-out\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.797660 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.797643    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39dfc855-2987-4dab-b0ca-8225ad97d1d9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.797951 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.797827    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.800169 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.800126    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.800345 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.800328    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-config-volume\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.801021 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.800977    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.802302 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.802277    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39dfc855-2987-4dab-b0ca-8225ad97d1d9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.802421 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.802282    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.802486 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.802448    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.803008 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.802971    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:18.803364 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.803347    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-web-config\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.806743 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.806702    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmgq4\" (UniqueName: \"kubernetes.io/projected/39dfc855-2987-4dab-b0ca-8225ad97d1d9-kube-api-access-hmgq4\") pod \"alertmanager-main-0\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.895769 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.895736    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret\") pod \"global-pull-secret-syncer-2z2wd\" (UID: \"bc73ab72-c095-4e0d-bbd9-b5296cadc187\") " pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:34:18.895941 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.895778    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs\") pod \"network-metrics-daemon-x7hnn\" (UID: \"fab79c82-8579-4ce6-9410-89fce50e5b70\") " pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:34:18.896930 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.896903    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:18.899592 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.899576    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 05 17:34:18.899640 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.899627    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 05 17:34:18.909280 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.909253    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fab79c82-8579-4ce6-9410-89fce50e5b70-metrics-certs\") pod \"network-metrics-daemon-x7hnn\" (UID: \"fab79c82-8579-4ce6-9410-89fce50e5b70\") " pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:34:18.909394 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.909311    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc73ab72-c095-4e0d-bbd9-b5296cadc187-original-pull-secret\") pod \"global-pull-secret-syncer-2z2wd\" (UID: \"bc73ab72-c095-4e0d-bbd9-b5296cadc187\") " pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:34:18.935571 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.935543    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:34:18.956314 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.956279    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2wd"
Aug 05 17:34:18.972599 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.972382    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bbv86\""
Aug 05 17:34:18.979834 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.979765    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7hnn"
Aug 05 17:34:18.997406 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:18.997107    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqqc4\" (UniqueName: \"kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4\") pod \"network-check-target-9vkxl\" (UID: \"e538a505-e584-496e-9907-e2fb84f0e872\") " pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:34:19.000203 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.000018    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 05 17:34:19.011213 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.010971    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 05 17:34:19.023503 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.023473    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqqc4\" (UniqueName: \"kubernetes.io/projected/e538a505-e584-496e-9907-e2fb84f0e872-kube-api-access-xqqc4\") pod \"network-check-target-9vkxl\" (UID: \"e538a505-e584-496e-9907-e2fb84f0e872\") " pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:34:19.156878 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.156837    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 05 17:34:19.162933 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.162904    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2z2wd"]
Aug 05 17:34:19.171755 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.171728    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x7hnn"]
Aug 05 17:34:19.265359 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.265278    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dt589\""
Aug 05 17:34:19.273606 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.273572    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:34:19.416578 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:34:19.416549    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc73ab72_c095_4e0d_bbd9_b5296cadc187.slice/crio-32e664e90c1e5aa0226ed08cfd9623e0d907091d59addd79de80d31073688edd WatchSource:0}: Error finding container 32e664e90c1e5aa0226ed08cfd9623e0d907091d59addd79de80d31073688edd: Status 404 returned error can't find the container with id 32e664e90c1e5aa0226ed08cfd9623e0d907091d59addd79de80d31073688edd
Aug 05 17:34:19.417277 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:34:19.417251    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39dfc855_2987_4dab_b0ca_8225ad97d1d9.slice/crio-fa4e736bd303a3f97f36c7a5f506e5a994aaca1bca17c0fb122476abee894b2d WatchSource:0}: Error finding container fa4e736bd303a3f97f36c7a5f506e5a994aaca1bca17c0fb122476abee894b2d: Status 404 returned error can't find the container with id fa4e736bd303a3f97f36c7a5f506e5a994aaca1bca17c0fb122476abee894b2d
Aug 05 17:34:19.417871 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:34:19.417838    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfab79c82_8579_4ce6_9410_89fce50e5b70.slice/crio-19141265ee48020aa11c609abfb52efc06ebd21b48a2b8fda6ff119642954c3f WatchSource:0}: Error finding container 19141265ee48020aa11c609abfb52efc06ebd21b48a2b8fda6ff119642954c3f: Status 404 returned error can't find the container with id 19141265ee48020aa11c609abfb52efc06ebd21b48a2b8fda6ff119642954c3f
Aug 05 17:34:19.551679 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.551555    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2z2wd" event={"ID":"bc73ab72-c095-4e0d-bbd9-b5296cadc187","Type":"ContainerStarted","Data":"32e664e90c1e5aa0226ed08cfd9623e0d907091d59addd79de80d31073688edd"}
Aug 05 17:34:19.553519 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.553489    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39dfc855-2987-4dab-b0ca-8225ad97d1d9","Type":"ContainerStarted","Data":"fa4e736bd303a3f97f36c7a5f506e5a994aaca1bca17c0fb122476abee894b2d"}
Aug 05 17:34:19.555691 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.555664    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x7hnn" event={"ID":"fab79c82-8579-4ce6-9410-89fce50e5b70","Type":"ContainerStarted","Data":"19141265ee48020aa11c609abfb52efc06ebd21b48a2b8fda6ff119642954c3f"}
Aug 05 17:34:19.616451 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.616423    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-d896f6548-kx6m8"]
Aug 05 17:34:19.619374 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.619346    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.621981 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.621959    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Aug 05 17:34:19.622338 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.622314    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-9br9z\""
Aug 05 17:34:19.622690 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.622670    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-3j67sc0unap15\""
Aug 05 17:34:19.622908 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.622890    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Aug 05 17:34:19.623025 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.622916    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Aug 05 17:34:19.623025 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.622994    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Aug 05 17:34:19.623143 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.623109    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Aug 05 17:34:19.635038 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.635010    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d896f6548-kx6m8"]
Aug 05 17:34:19.705867 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.705837    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5221e25b-9034-4392-a945-2fce3cb7f40e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.706054 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.705880    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxzhr\" (UniqueName: \"kubernetes.io/projected/5221e25b-9034-4392-a945-2fce3cb7f40e-kube-api-access-lxzhr\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.706054 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.705931    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5221e25b-9034-4392-a945-2fce3cb7f40e-metrics-client-ca\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.706054 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.705998    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5221e25b-9034-4392-a945-2fce3cb7f40e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.706054 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.706050    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5221e25b-9034-4392-a945-2fce3cb7f40e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.706305 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.706095    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5221e25b-9034-4392-a945-2fce3cb7f40e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.706305 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.706120    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5221e25b-9034-4392-a945-2fce3cb7f40e-secret-grpc-tls\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.706305 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.706152    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5221e25b-9034-4392-a945-2fce3cb7f40e-secret-thanos-querier-tls\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.807363 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.806749    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5221e25b-9034-4392-a945-2fce3cb7f40e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.807363 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.806794    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5221e25b-9034-4392-a945-2fce3cb7f40e-secret-grpc-tls\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.807363 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.806831    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5221e25b-9034-4392-a945-2fce3cb7f40e-secret-thanos-querier-tls\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.807363 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.806866    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5221e25b-9034-4392-a945-2fce3cb7f40e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.807363 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.806895    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxzhr\" (UniqueName: \"kubernetes.io/projected/5221e25b-9034-4392-a945-2fce3cb7f40e-kube-api-access-lxzhr\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.807363 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.806944    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5221e25b-9034-4392-a945-2fce3cb7f40e-metrics-client-ca\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.807363 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.806976    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5221e25b-9034-4392-a945-2fce3cb7f40e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.807363 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.807028    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5221e25b-9034-4392-a945-2fce3cb7f40e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.809017 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.808359    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5221e25b-9034-4392-a945-2fce3cb7f40e-metrics-client-ca\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.810742 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.810680    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5221e25b-9034-4392-a945-2fce3cb7f40e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.811719 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.811693    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5221e25b-9034-4392-a945-2fce3cb7f40e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.812704 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.812593    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5221e25b-9034-4392-a945-2fce3cb7f40e-secret-grpc-tls\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.813769 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.813709    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5221e25b-9034-4392-a945-2fce3cb7f40e-secret-thanos-querier-tls\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.813868 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.813796    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5221e25b-9034-4392-a945-2fce3cb7f40e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.813868 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.813827    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5221e25b-9034-4392-a945-2fce3cb7f40e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.820217 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.820167    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxzhr\" (UniqueName: \"kubernetes.io/projected/5221e25b-9034-4392-a945-2fce3cb7f40e-kube-api-access-lxzhr\") pod \"thanos-querier-d896f6548-kx6m8\" (UID: \"5221e25b-9034-4392-a945-2fce3cb7f40e\") " pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.842448 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.842427    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9vkxl"]
Aug 05 17:34:19.930464 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.930394    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:19.931371 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.931346    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:19.941129 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.941102    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6499c57964-wvz2d"]
Aug 05 17:34:19.943813 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.943792    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:19.952585 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.952555    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Aug 05 17:34:19.954316 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.954287    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6499c57964-wvz2d"]
Aug 05 17:34:19.996261 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:19.996227    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:20.009407 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.009378    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-trusted-ca-bundle\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.009561 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.009431    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b30958f2-4b26-4698-9a9f-bdb8ac268265-console-serving-cert\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.009561 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.009470    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-console-config\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.009561 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.009493    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxwss\" (UniqueName: \"kubernetes.io/projected/b30958f2-4b26-4698-9a9f-bdb8ac268265-kube-api-access-gxwss\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.009561 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.009521    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b30958f2-4b26-4698-9a9f-bdb8ac268265-console-oauth-config\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.009561 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.009543    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-service-ca\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.009819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.009611    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-oauth-serving-cert\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.036970 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:34:20.036930    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode538a505_e584_496e_9907_e2fb84f0e872.slice/crio-e456a77a2cdd21143cf26df15d0577150b36a88e0d78d5e2bd92c1095deb46fe WatchSource:0}: Error finding container e456a77a2cdd21143cf26df15d0577150b36a88e0d78d5e2bd92c1095deb46fe: Status 404 returned error can't find the container with id e456a77a2cdd21143cf26df15d0577150b36a88e0d78d5e2bd92c1095deb46fe
Aug 05 17:34:20.110986 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.110961    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-oauth-serving-cert\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.111087 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.111015    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-trusted-ca-bundle\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.111087 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.111055    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b30958f2-4b26-4698-9a9f-bdb8ac268265-console-serving-cert\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.111221 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.111090    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-console-config\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.111221 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.111115    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxwss\" (UniqueName: \"kubernetes.io/projected/b30958f2-4b26-4698-9a9f-bdb8ac268265-kube-api-access-gxwss\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.111329 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.111261    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b30958f2-4b26-4698-9a9f-bdb8ac268265-console-oauth-config\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.111329 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.111291    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-service-ca\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.111832 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.111791    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-oauth-serving-cert\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.112067 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.112042    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-service-ca\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.112198 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.112100    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-console-config\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.112198 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.112124    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-trusted-ca-bundle\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.114160 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.114140    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b30958f2-4b26-4698-9a9f-bdb8ac268265-console-serving-cert\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.114977 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.114922    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b30958f2-4b26-4698-9a9f-bdb8ac268265-console-oauth-config\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.122739 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.122712    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxwss\" (UniqueName: \"kubernetes.io/projected/b30958f2-4b26-4698-9a9f-bdb8ac268265-kube-api-access-gxwss\") pod \"console-6499c57964-wvz2d\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") " pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.210116 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.209576    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d896f6548-kx6m8"]
Aug 05 17:34:20.215445 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:34:20.215403    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5221e25b_9034_4392_a945_2fce3cb7f40e.slice/crio-9bf382812ff0d0401a3b73ace0e554eb4a2c458f2a190e84c818bb780c3442d9 WatchSource:0}: Error finding container 9bf382812ff0d0401a3b73ace0e554eb4a2c458f2a190e84c818bb780c3442d9: Status 404 returned error can't find the container with id 9bf382812ff0d0401a3b73ace0e554eb4a2c458f2a190e84c818bb780c3442d9
Aug 05 17:34:20.259876 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.259837    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:20.451868 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.451790    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6499c57964-wvz2d"]
Aug 05 17:34:20.456137 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:34:20.456102    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb30958f2_4b26_4698_9a9f_bdb8ac268265.slice/crio-9c5942d94bd2e0473e420979b1c721942726947238a45b578a1d7e2bf1242158 WatchSource:0}: Error finding container 9c5942d94bd2e0473e420979b1c721942726947238a45b578a1d7e2bf1242158: Status 404 returned error can't find the container with id 9c5942d94bd2e0473e420979b1c721942726947238a45b578a1d7e2bf1242158
Aug 05 17:34:20.463808 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.463778    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:20.515793 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.515383    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ntkvz"
Aug 05 17:34:20.523621 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.523545    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:20.562994 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.562915    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6499c57964-wvz2d" event={"ID":"b30958f2-4b26-4698-9a9f-bdb8ac268265","Type":"ContainerStarted","Data":"9c5942d94bd2e0473e420979b1c721942726947238a45b578a1d7e2bf1242158"}
Aug 05 17:34:20.566408 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.566212    2569 generic.go:356] "Generic (PLEG): container finished" podID="2c558452-bf1d-44ba-a169-a78f38dc4bb3" containerID="3e671def81de940b1c8874712e5b0818266fca74fe60dffcafaeb46fe060ab8f" exitCode=0
Aug 05 17:34:20.566408 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.566306    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w69vr" event={"ID":"2c558452-bf1d-44ba-a169-a78f38dc4bb3","Type":"ContainerDied","Data":"3e671def81de940b1c8874712e5b0818266fca74fe60dffcafaeb46fe060ab8f"}
Aug 05 17:34:20.568529 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.568498    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9vkxl" event={"ID":"e538a505-e584-496e-9907-e2fb84f0e872","Type":"ContainerStarted","Data":"e456a77a2cdd21143cf26df15d0577150b36a88e0d78d5e2bd92c1095deb46fe"}
Aug 05 17:34:20.574022 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.573711    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm" event={"ID":"8721c842-6692-47bd-a37a-3c1d48edc59a","Type":"ContainerStarted","Data":"5d45b5675a05970832c262732af581d5259593efb76eae14ae625909133a6a53"}
Aug 05 17:34:20.574022 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.573763    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm" event={"ID":"8721c842-6692-47bd-a37a-3c1d48edc59a","Type":"ContainerStarted","Data":"2ebd40874a98584df2d49150fd7dac696bec53545ee6c027354a0cf6e5c5e14d"}
Aug 05 17:34:20.580546 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.574274    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm" event={"ID":"8721c842-6692-47bd-a37a-3c1d48edc59a","Type":"ContainerStarted","Data":"b353b2c5c21a2e1b835388233a0301bd35638ccbf541c729ecdf2f3a57cf93d1"}
Aug 05 17:34:20.592142 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.591983    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz" event={"ID":"9ee151b3-b9ea-499f-afd4-816e849c1c5f","Type":"ContainerStarted","Data":"edaaf3bf45d16fc8cb3755287e37ff8127bb738dde1df1d16c44396dcffd7522"}
Aug 05 17:34:20.593681 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.593585    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8" event={"ID":"5221e25b-9034-4392-a945-2fce3cb7f40e","Type":"ContainerStarted","Data":"9bf382812ff0d0401a3b73ace0e554eb4a2c458f2a190e84c818bb780c3442d9"}
Aug 05 17:34:20.609598 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.609559    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-855bm" podStartSLOduration=1.9084539820000002 podStartE2EDuration="3.609541859s" totalImagesPullingTime="1.701087877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:34:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:20.607570943 +0000 UTC m=+67.960245541" watchObservedRunningTime="2026-08-05 17:34:20.609541859 +0000 UTC m=+67.962216434"
Aug 05 17:34:20.632923 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.632190    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-vmbsz" podStartSLOduration=2.049256148 podStartE2EDuration="3.632152538s" totalImagesPullingTime="1.58289639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:34:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:20.631554877 +0000 UTC m=+67.984229484" watchObservedRunningTime="2026-08-05 17:34:20.632152538 +0000 UTC m=+67.984827134"
Aug 05 17:34:20.785622 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.785589    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:20.940612 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:20.940531    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:21.602747 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.602713    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w69vr" event={"ID":"2c558452-bf1d-44ba-a169-a78f38dc4bb3","Type":"ContainerStarted","Data":"4a587cb03ed5d4a6d0770c5d7212b6f55ec5f24ee0d247e8914961ed9a198ff9"}
Aug 05 17:34:21.603231 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.602784    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w69vr" event={"ID":"2c558452-bf1d-44ba-a169-a78f38dc4bb3","Type":"ContainerStarted","Data":"d3f6f9201b736ab77a305a425bcb5ca7f623b1018986ee5ca8b1b225b4f86aed"}
Aug 05 17:34:21.607148 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.606417    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6499c57964-wvz2d" event={"ID":"b30958f2-4b26-4698-9a9f-bdb8ac268265","Type":"ContainerStarted","Data":"2fd1cb14757a9e28bc1dbb15ad246da349acaac01b545e8497036aeaaaa4275f"}
Aug 05 17:34:21.650986 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.649717    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-w69vr" podStartSLOduration=2.88298615 podStartE2EDuration="4.649695982s" totalImagesPullingTime="1.66141556s" totalInitContainerRuntime="105.294272ms" isStatefulPod=true podCreationTimestamp="2026-08-05 17:34:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:21.627692579 +0000 UTC m=+68.980367175" watchObservedRunningTime="2026-08-05 17:34:21.649695982 +0000 UTC m=+69.002370575"
Aug 05 17:34:21.650986 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.649872    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6499c57964-wvz2d" podStartSLOduration=2.649862518 podStartE2EDuration="2.649862518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:34:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:21.648540392 +0000 UTC m=+69.001214987" watchObservedRunningTime="2026-08-05 17:34:21.649862518 +0000 UTC m=+69.002537114"
Aug 05 17:34:21.772056 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.771889    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-798899d854-2rwbz"]
Aug 05 17:34:21.792481 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.792454    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-798899d854-2rwbz"]
Aug 05 17:34:21.792643 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.792618    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.797676 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.796929    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Aug 05 17:34:21.797676 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.796968    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-cc8rj\""
Aug 05 17:34:21.797676 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.797285    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Aug 05 17:34:21.797676 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.797317    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7uuq9o0c7eqrh\""
Aug 05 17:34:21.797676 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.797520    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Aug 05 17:34:21.797676 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.797633    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Aug 05 17:34:21.828301 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.828272    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n89xg\" (UniqueName: \"kubernetes.io/projected/81a88269-e4c5-4800-9f53-45cee2b243bc-kube-api-access-n89xg\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.828443 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.828334    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/81a88269-e4c5-4800-9f53-45cee2b243bc-metrics-server-audit-profiles\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.828443 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.828395    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/81a88269-e4c5-4800-9f53-45cee2b243bc-secret-metrics-server-tls\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.828443 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.828426    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/81a88269-e4c5-4800-9f53-45cee2b243bc-audit-log\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.828605 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.828464    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81a88269-e4c5-4800-9f53-45cee2b243bc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.828605 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.828495    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/81a88269-e4c5-4800-9f53-45cee2b243bc-client-ca-bundle\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.828605 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.828525    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/81a88269-e4c5-4800-9f53-45cee2b243bc-secret-metrics-server-client-certs\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.929667 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.929589    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n89xg\" (UniqueName: \"kubernetes.io/projected/81a88269-e4c5-4800-9f53-45cee2b243bc-kube-api-access-n89xg\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.929667 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.929659    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/81a88269-e4c5-4800-9f53-45cee2b243bc-metrics-server-audit-profiles\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.929870 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.929689    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/81a88269-e4c5-4800-9f53-45cee2b243bc-secret-metrics-server-tls\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.929870 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.929716    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/81a88269-e4c5-4800-9f53-45cee2b243bc-audit-log\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.929870 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.929756    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81a88269-e4c5-4800-9f53-45cee2b243bc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.929870 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.929785    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/81a88269-e4c5-4800-9f53-45cee2b243bc-client-ca-bundle\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.929870 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.929811    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/81a88269-e4c5-4800-9f53-45cee2b243bc-secret-metrics-server-client-certs\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.930353 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.930325    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/81a88269-e4c5-4800-9f53-45cee2b243bc-audit-log\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.930593 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.930546    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81a88269-e4c5-4800-9f53-45cee2b243bc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.931199 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.931152    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/81a88269-e4c5-4800-9f53-45cee2b243bc-metrics-server-audit-profiles\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.932957 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.932926    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/81a88269-e4c5-4800-9f53-45cee2b243bc-secret-metrics-server-client-certs\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.937117 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.937094    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/81a88269-e4c5-4800-9f53-45cee2b243bc-client-ca-bundle\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.939260 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.939238    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n89xg\" (UniqueName: \"kubernetes.io/projected/81a88269-e4c5-4800-9f53-45cee2b243bc-kube-api-access-n89xg\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:21.945618 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:21.945579    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/81a88269-e4c5-4800-9f53-45cee2b243bc-secret-metrics-server-tls\") pod \"metrics-server-798899d854-2rwbz\" (UID: \"81a88269-e4c5-4800-9f53-45cee2b243bc\") " pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:22.053608 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.053570    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:22.108448 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.108421    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:22.154833 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.154717    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:22.245137 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.245099    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-pvjkg"]
Aug 05 17:34:22.250158 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.250086    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:22.252055 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.251086    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pvjkg"
Aug 05 17:34:22.256530 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.256076    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Aug 05 17:34:22.256530 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.256354    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2kh26\""
Aug 05 17:34:22.268365 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.267315    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f86985667-9gbwx"]
Aug 05 17:34:22.270697 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.270675    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-pvjkg"]
Aug 05 17:34:22.293681 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.293657    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69c65ccfc5-z7nkj"]
Aug 05 17:34:22.296775 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.296751    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.310512 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.310150    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-798899d854-2rwbz"]
Aug 05 17:34:22.313781 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.313603    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69c65ccfc5-z7nkj"]
Aug 05 17:34:22.334828 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.334401    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-service-ca\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.334828 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.334455    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-console-config\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.334828 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.334510    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20caaf9f-d0d1-43f7-a283-941707f60e40-console-oauth-config\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.334828 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.334543    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20caaf9f-d0d1-43f7-a283-941707f60e40-console-serving-cert\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.334828 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.334569    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-trusted-ca-bundle\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.334828 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.334593    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-oauth-serving-cert\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.334828 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.334617    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624e4dca-7205-45fa-b44d-5d09f307468f-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-pvjkg\" (UID: \"624e4dca-7205-45fa-b44d-5d09f307468f\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pvjkg"
Aug 05 17:34:22.334828 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.334656    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7qfc\" (UniqueName: \"kubernetes.io/projected/624e4dca-7205-45fa-b44d-5d09f307468f-kube-api-access-w7qfc\") pod \"monitoring-plugin-6dffbbd67-pvjkg\" (UID: \"624e4dca-7205-45fa-b44d-5d09f307468f\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pvjkg"
Aug 05 17:34:22.334828 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.334695    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2z9z\" (UniqueName: \"kubernetes.io/projected/20caaf9f-d0d1-43f7-a283-941707f60e40-kube-api-access-r2z9z\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.341120 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.341097    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:22.435094 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.435030    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-service-ca\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.435094 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.435067    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-console-config\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.435336 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.435102    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20caaf9f-d0d1-43f7-a283-941707f60e40-console-oauth-config\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.435336 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.435129    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20caaf9f-d0d1-43f7-a283-941707f60e40-console-serving-cert\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.435336 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.435153    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-trusted-ca-bundle\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.435336 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.435202    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-oauth-serving-cert\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.435336 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.435229    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624e4dca-7205-45fa-b44d-5d09f307468f-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-pvjkg\" (UID: \"624e4dca-7205-45fa-b44d-5d09f307468f\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pvjkg"
Aug 05 17:34:22.435336 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.435280    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7qfc\" (UniqueName: \"kubernetes.io/projected/624e4dca-7205-45fa-b44d-5d09f307468f-kube-api-access-w7qfc\") pod \"monitoring-plugin-6dffbbd67-pvjkg\" (UID: \"624e4dca-7205-45fa-b44d-5d09f307468f\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pvjkg"
Aug 05 17:34:22.435336 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.435313    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2z9z\" (UniqueName: \"kubernetes.io/projected/20caaf9f-d0d1-43f7-a283-941707f60e40-kube-api-access-r2z9z\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.435799 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.435776    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-service-ca\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.435864 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.435840    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-console-config\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.435960 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:34:22.435945    2569 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Aug 05 17:34:22.436034 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:34:22.436019    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/624e4dca-7205-45fa-b44d-5d09f307468f-monitoring-plugin-cert podName:624e4dca-7205-45fa-b44d-5d09f307468f nodeName:}" failed. No retries permitted until 2026-08-05 17:34:22.936000864 +0000 UTC m=+70.288675447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/624e4dca-7205-45fa-b44d-5d09f307468f-monitoring-plugin-cert") pod "monitoring-plugin-6dffbbd67-pvjkg" (UID: "624e4dca-7205-45fa-b44d-5d09f307468f") : secret "monitoring-plugin-cert" not found
Aug 05 17:34:22.436370 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.436348    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-oauth-serving-cert\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.437096 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.437045    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-trusted-ca-bundle\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.439088 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.439064    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20caaf9f-d0d1-43f7-a283-941707f60e40-console-serving-cert\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.439661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.439637    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20caaf9f-d0d1-43f7-a283-941707f60e40-console-oauth-config\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.449507 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.449467    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2z9z\" (UniqueName: \"kubernetes.io/projected/20caaf9f-d0d1-43f7-a283-941707f60e40-kube-api-access-r2z9z\") pod \"console-69c65ccfc5-z7nkj\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") " pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.452724 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.452698    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7qfc\" (UniqueName: \"kubernetes.io/projected/624e4dca-7205-45fa-b44d-5d09f307468f-kube-api-access-w7qfc\") pod \"monitoring-plugin-6dffbbd67-pvjkg\" (UID: \"624e4dca-7205-45fa-b44d-5d09f307468f\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pvjkg"
Aug 05 17:34:22.610948 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.610758    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:22.613481 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.613454    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x7hnn" event={"ID":"fab79c82-8579-4ce6-9410-89fce50e5b70","Type":"ContainerStarted","Data":"62bf3c52fa8004f30773f580a965dfb1c3a375c62ec480d7f13b8b60b2053714"}
Aug 05 17:34:22.613595 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.613514    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x7hnn" event={"ID":"fab79c82-8579-4ce6-9410-89fce50e5b70","Type":"ContainerStarted","Data":"cd78d69ac8c502e9d76773d89bbc3cf50f0019c3948bfc3e17dd587ae5ed5ca5"}
Aug 05 17:34:22.617868 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.616688    2569 generic.go:356] "Generic (PLEG): container finished" podID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerID="aed68aabdd6955c9123d91b756a4dbc145bf488c44f804199d365850267414b9" exitCode=0
Aug 05 17:34:22.617868 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.616814    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39dfc855-2987-4dab-b0ca-8225ad97d1d9","Type":"ContainerDied","Data":"aed68aabdd6955c9123d91b756a4dbc145bf488c44f804199d365850267414b9"}
Aug 05 17:34:22.622908 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.622888    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:22.633032 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.632956    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-x7hnn" podStartSLOduration=67.114966177 podStartE2EDuration="1m9.63293783s" totalImagesPullingTime="2.517971653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:33:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:22.632348239 +0000 UTC m=+69.985022870" watchObservedRunningTime="2026-08-05 17:34:22.63293783 +0000 UTC m=+69.985612438"
Aug 05 17:34:22.763029 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.762949    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:22.836881 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.836849    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:22.941426 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.941386    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624e4dca-7205-45fa-b44d-5d09f307468f-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-pvjkg\" (UID: \"624e4dca-7205-45fa-b44d-5d09f307468f\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pvjkg"
Aug 05 17:34:22.944849 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:22.944824    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624e4dca-7205-45fa-b44d-5d09f307468f-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-pvjkg\" (UID: \"624e4dca-7205-45fa-b44d-5d09f307468f\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pvjkg"
Aug 05 17:34:23.026512 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:34:23.026440    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81a88269_e4c5_4800_9f53_45cee2b243bc.slice/crio-aa2c15a6c0bd8945a642a855795cf52c1340ac30583928c95e55d28abcafb1d7 WatchSource:0}: Error finding container aa2c15a6c0bd8945a642a855795cf52c1340ac30583928c95e55d28abcafb1d7: Status 404 returned error can't find the container with id aa2c15a6c0bd8945a642a855795cf52c1340ac30583928c95e55d28abcafb1d7
Aug 05 17:34:23.173528 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.173491    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pvjkg"
Aug 05 17:34:23.625148 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.625106    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-798899d854-2rwbz" event={"ID":"81a88269-e4c5-4800-9f53-45cee2b243bc","Type":"ContainerStarted","Data":"aa2c15a6c0bd8945a642a855795cf52c1340ac30583928c95e55d28abcafb1d7"}
Aug 05 17:34:23.730601 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.730528    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:23.736516 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.734402    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 05 17:34:23.738237 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.738216    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.742938 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.741018    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Aug 05 17:34:23.742938 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.741051    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Aug 05 17:34:23.742938 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.741293    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Aug 05 17:34:23.742938 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.741319    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cojeqbt7sddc7\""
Aug 05 17:34:23.743522 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.743505    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Aug 05 17:34:23.743706 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.743686    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Aug 05 17:34:23.743873 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.743860    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-b2nbt\""
Aug 05 17:34:23.744065 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.744049    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Aug 05 17:34:23.744441 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.744428    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Aug 05 17:34:23.744646 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.744633    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Aug 05 17:34:23.744818 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.744805    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Aug 05 17:34:23.745944 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.745909    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Aug 05 17:34:23.746124 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.746112    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Aug 05 17:34:23.746306 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.746289    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Aug 05 17:34:23.748698 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.748649    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Aug 05 17:34:23.753207 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.752792    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Aug 05 17:34:23.756617 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.756337    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 05 17:34:23.850740 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.850706    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.850740 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.850741    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.850980 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.850791    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/132dda5b-3438-4121-90d5-b6f71da27477-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.850980 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.850812    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.850980 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.850836    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.850980 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.850885    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.850980 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.850906    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.850980 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.850923    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.850980 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.850953    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-config\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.850980 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.850977    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.851389 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.850997    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.851389 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.851021    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.851389 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.851047    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.851389 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.851076    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.851389 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.851094    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.851389 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.851115    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/132dda5b-3438-4121-90d5-b6f71da27477-config-out\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.851389 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.851137    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqcc2\" (UniqueName: \"kubernetes.io/projected/132dda5b-3438-4121-90d5-b6f71da27477-kube-api-access-qqcc2\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.851389 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.851165    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-web-config\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.851389 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.851227    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.851389 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.851302    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.952387 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.952296    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqcc2\" (UniqueName: \"kubernetes.io/projected/132dda5b-3438-4121-90d5-b6f71da27477-kube-api-access-qqcc2\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.952387 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.952347    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-web-config\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.952387 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.952381    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.952646 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.952418    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.952646 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.952469    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.952646 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.952493    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.952646 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.952545    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/132dda5b-3438-4121-90d5-b6f71da27477-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.952646 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.952569    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.952646 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.952591    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.952646 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.952642    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.953025 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.952665    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.953025 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.952690    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.953025 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.952721    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-config\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.953025 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.952767    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.953025 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.952790    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.953025 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.952819    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.953025 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.952843    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.953025 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.952878    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.953469 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.953440    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.953639 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.953614    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.953728 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.953663    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/132dda5b-3438-4121-90d5-b6f71da27477-config-out\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.954730 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.953879    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.954730 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.954013    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.954730 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.954435    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.954942 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.954778    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.956249 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.955158    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.956249 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.955722    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.956778 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.956727    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.957082 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.957056    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-web-config\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.958447 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.958405    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.959576 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.959547    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.960033 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.959990    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/132dda5b-3438-4121-90d5-b6f71da27477-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.962118 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.961944    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqcc2\" (UniqueName: \"kubernetes.io/projected/132dda5b-3438-4121-90d5-b6f71da27477-kube-api-access-qqcc2\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.962227 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.962131    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-config\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.962612 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.962591    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.962841 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.962820    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.962909 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.962850    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/132dda5b-3438-4121-90d5-b6f71da27477-config-out\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.963264 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.963239    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.963957 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.963935    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:23.964778 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:23.964754    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:24.055564 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:24.055515    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:24.487453 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:24.487416    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:24.685234 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:24.685201    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:24.790672 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:24.790526    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:25.674493 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:25.674429    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:25.741489 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:25.741453    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:25.771828 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:25.771803    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69c65ccfc5-z7nkj"]
Aug 05 17:34:26.014642 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:34:26.014569    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20caaf9f_d0d1_43f7_a283_941707f60e40.slice/crio-53ed9016c68ad3e4a67bb02b0db649b8b1d137b465b9be58c6aca18e62f75d7c WatchSource:0}: Error finding container 53ed9016c68ad3e4a67bb02b0db649b8b1d137b465b9be58c6aca18e62f75d7c: Status 404 returned error can't find the container with id 53ed9016c68ad3e4a67bb02b0db649b8b1d137b465b9be58c6aca18e62f75d7c
Aug 05 17:34:26.247668 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.247629    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-pvjkg"]
Aug 05 17:34:26.252348 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:34:26.252306    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod624e4dca_7205_45fa_b44d_5d09f307468f.slice/crio-72d975d52eed111ae819f7600a92c398d2e3e1038991de3ef0ebc58548b5566c WatchSource:0}: Error finding container 72d975d52eed111ae819f7600a92c398d2e3e1038991de3ef0ebc58548b5566c: Status 404 returned error can't find the container with id 72d975d52eed111ae819f7600a92c398d2e3e1038991de3ef0ebc58548b5566c
Aug 05 17:34:26.277664 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.277341    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 05 17:34:26.641083 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.641037    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pvjkg" event={"ID":"624e4dca-7205-45fa-b44d-5d09f307468f","Type":"ContainerStarted","Data":"72d975d52eed111ae819f7600a92c398d2e3e1038991de3ef0ebc58548b5566c"}
Aug 05 17:34:26.642928 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.642902    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69c65ccfc5-z7nkj" event={"ID":"20caaf9f-d0d1-43f7-a283-941707f60e40","Type":"ContainerStarted","Data":"87d647fb8afa3970204e2c4c40468f7b2227504373279a4e13c772c1730a4c0d"}
Aug 05 17:34:26.643059 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.642949    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69c65ccfc5-z7nkj" event={"ID":"20caaf9f-d0d1-43f7-a283-941707f60e40","Type":"ContainerStarted","Data":"53ed9016c68ad3e4a67bb02b0db649b8b1d137b465b9be58c6aca18e62f75d7c"}
Aug 05 17:34:26.644776 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.644747    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9vkxl" event={"ID":"e538a505-e584-496e-9907-e2fb84f0e872","Type":"ContainerStarted","Data":"65e675bc61da9efcc9b72d7a6570a61ebc712138a76b14faadd07846c9477ce0"}
Aug 05 17:34:26.644933 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.644799    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:34:26.646460 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.646436    2569 generic.go:356] "Generic (PLEG): container finished" podID="132dda5b-3438-4121-90d5-b6f71da27477" containerID="02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048" exitCode=0
Aug 05 17:34:26.646596 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.646525    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"132dda5b-3438-4121-90d5-b6f71da27477","Type":"ContainerDied","Data":"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048"}
Aug 05 17:34:26.646596 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.646586    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"132dda5b-3438-4121-90d5-b6f71da27477","Type":"ContainerStarted","Data":"a7be02f7ce84f5674498ed1ce74237e2692ffb206aa213ad59365925bb336a76"}
Aug 05 17:34:26.648309 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.648244    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2z2wd" event={"ID":"bc73ab72-c095-4e0d-bbd9-b5296cadc187","Type":"ContainerStarted","Data":"3e47093f90961af7f52097be45e86bf303bf4daace906abd58a62ee5fc19ee35"}
Aug 05 17:34:26.651735 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.651716    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39dfc855-2987-4dab-b0ca-8225ad97d1d9","Type":"ContainerStarted","Data":"3863d34bf80e7bba08ce9cbb653f3bd1b3c95bde6f03eb293bf9b6dc2f79c4b0"}
Aug 05 17:34:26.651828 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.651779    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39dfc855-2987-4dab-b0ca-8225ad97d1d9","Type":"ContainerStarted","Data":"3b3ab342fb9d26aea67f566c4ebfed5a40972a99ef5c88002839a87be2bd51dc"}
Aug 05 17:34:26.651828 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.651805    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39dfc855-2987-4dab-b0ca-8225ad97d1d9","Type":"ContainerStarted","Data":"7598b1469bb555c351f448b7a98f0cce9a62b5187e27f49f612f61340928a407"}
Aug 05 17:34:26.651938 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.651843    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39dfc855-2987-4dab-b0ca-8225ad97d1d9","Type":"ContainerStarted","Data":"004b37dd2cfcb97a4a70a9ea21161ece646d21e6835b540a128861264b7e4be9"}
Aug 05 17:34:26.651938 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.651884    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39dfc855-2987-4dab-b0ca-8225ad97d1d9","Type":"ContainerStarted","Data":"e70e7bb5b3b1c5292e566e7ed6b9d63b7fcb4bd3c5e95fa6ba86218d296b558a"}
Aug 05 17:34:26.654062 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.654044    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8" event={"ID":"5221e25b-9034-4392-a945-2fce3cb7f40e","Type":"ContainerStarted","Data":"6a67b9f02a7e2f51dc9788310137e349ca58d8b999e12629dcdb982f0b9a60fd"}
Aug 05 17:34:26.654143 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.654095    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8" event={"ID":"5221e25b-9034-4392-a945-2fce3cb7f40e","Type":"ContainerStarted","Data":"a041571b4f467490507a3ea687d26ec0d495d15910649215f88dacb1d3d57917"}
Aug 05 17:34:26.654143 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.654121    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8" event={"ID":"5221e25b-9034-4392-a945-2fce3cb7f40e","Type":"ContainerStarted","Data":"fde0319878d7d77ff2a12fd1be7ac3631be0f5da415bd56f75327f3338fbaf44"}
Aug 05 17:34:26.663575 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.663527    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69c65ccfc5-z7nkj" podStartSLOduration=4.663510142 podStartE2EDuration="4.663510142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:34:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:26.662878009 +0000 UTC m=+74.015552628" watchObservedRunningTime="2026-08-05 17:34:26.663510142 +0000 UTC m=+74.016184737"
Aug 05 17:34:26.680631 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.680595    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2z2wd" podStartSLOduration=67.08063445 podStartE2EDuration="1m13.680577826s" totalImagesPullingTime="6.599943376s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:33:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:26.680063099 +0000 UTC m=+74.032737692" watchObservedRunningTime="2026-08-05 17:34:26.680577826 +0000 UTC m=+74.033252398"
Aug 05 17:34:26.704989 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:26.704937    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9vkxl" podStartSLOduration=67.726447539 podStartE2EDuration="1m13.704922505s" totalImagesPullingTime="5.978474966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:33:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:26.703603449 +0000 UTC m=+74.056278066" watchObservedRunningTime="2026-08-05 17:34:26.704922505 +0000 UTC m=+74.057597101"
Aug 05 17:34:27.665494 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:27.665402    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39dfc855-2987-4dab-b0ca-8225ad97d1d9","Type":"ContainerStarted","Data":"c638126fe5b50cb4e506b8cd4a3d5d2b26885bfb89fe1f89d8a6e409e2da3c2f"}
Aug 05 17:34:27.670223 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:27.670173    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8" event={"ID":"5221e25b-9034-4392-a945-2fce3cb7f40e","Type":"ContainerStarted","Data":"57932797ac35998c30b51676de2956f5b0ff8004b7cac1ae2ff98eb74f76ca1c"}
Aug 05 17:34:27.671743 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:27.671712    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-798899d854-2rwbz" event={"ID":"81a88269-e4c5-4800-9f53-45cee2b243bc","Type":"ContainerStarted","Data":"c992fa4dcf2b07c82384d18e9db61a7c3f0deb891aada0293117714dcce34bf5"}
Aug 05 17:34:27.710279 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:27.709275    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.7509333700000003 podStartE2EDuration="9.709255564s" totalImagesPullingTime="6.947588559s" totalInitContainerRuntime="10.733635ms" isStatefulPod=false podCreationTimestamp="2026-08-05 17:34:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:27.707645162 +0000 UTC m=+75.060319758" watchObservedRunningTime="2026-08-05 17:34:27.709255564 +0000 UTC m=+75.061930159"
Aug 05 17:34:27.744170 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:27.744114    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-798899d854-2rwbz" podStartSLOduration=2.759003333 podStartE2EDuration="6.744093904s" totalImagesPullingTime="3.985090571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:27.741590294 +0000 UTC m=+75.094264888" watchObservedRunningTime="2026-08-05 17:34:27.744093904 +0000 UTC m=+75.096768499"
Aug 05 17:34:27.756946 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:27.756916    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:27.851749 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:27.851718    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:28.679598 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:28.679565    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8" event={"ID":"5221e25b-9034-4392-a945-2fce3cb7f40e","Type":"ContainerStarted","Data":"2d22e5eb718640256cde398f008ba88ddaf4a4be489595eadf2083958158cd6d"}
Aug 05 17:34:28.680035 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:28.679671    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8" event={"ID":"5221e25b-9034-4392-a945-2fce3cb7f40e","Type":"ContainerStarted","Data":"7d0ea29ae1bdf2aed063d45987faaab9e2f718e28277cc928aa82a050e53ec2d"}
Aug 05 17:34:28.680035 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:28.679728    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:28.681227 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:28.681195    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pvjkg" event={"ID":"624e4dca-7205-45fa-b44d-5d09f307468f","Type":"ContainerStarted","Data":"c493e36ff989354d752edaec247b967593e451bab03fc9beeb04c575fc12aeb4"}
Aug 05 17:34:28.681343 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:28.681320    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pvjkg"
Aug 05 17:34:28.687224 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:28.687166    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pvjkg"
Aug 05 17:34:28.709498 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:28.709448    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8" podStartSLOduration=2.7184529939999997 podStartE2EDuration="9.709431677s" totalImagesPullingTime="6.990978683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:34:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:28.709394631 +0000 UTC m=+76.062069237" watchObservedRunningTime="2026-08-05 17:34:28.709431677 +0000 UTC m=+76.062106273"
Aug 05 17:34:28.731163 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:28.731121    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pvjkg" podStartSLOduration=4.842081069 podStartE2EDuration="6.731106586s" totalImagesPullingTime="1.889025517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:34:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:28.730018038 +0000 UTC m=+76.082692635" watchObservedRunningTime="2026-08-05 17:34:28.731106586 +0000 UTC m=+76.083781179"
Aug 05 17:34:29.871222 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:29.871163    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:29.971720 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:29.971688    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:30.072792 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:30.072760    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:30.260712 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:30.260689    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:30.260800 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:30.260751    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:30.266391 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:30.266244    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:30.692895 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:30.692813    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"132dda5b-3438-4121-90d5-b6f71da27477","Type":"ContainerStarted","Data":"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31"}
Aug 05 17:34:30.692895 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:30.692884    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"132dda5b-3438-4121-90d5-b6f71da27477","Type":"ContainerStarted","Data":"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518"}
Aug 05 17:34:30.693110 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:30.692925    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"132dda5b-3438-4121-90d5-b6f71da27477","Type":"ContainerStarted","Data":"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df"}
Aug 05 17:34:30.693110 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:30.692955    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"132dda5b-3438-4121-90d5-b6f71da27477","Type":"ContainerStarted","Data":"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad"}
Aug 05 17:34:30.693110 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:30.692979    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"132dda5b-3438-4121-90d5-b6f71da27477","Type":"ContainerStarted","Data":"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1"}
Aug 05 17:34:30.693110 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:30.693020    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"132dda5b-3438-4121-90d5-b6f71da27477","Type":"ContainerStarted","Data":"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb"}
Aug 05 17:34:30.696609 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:30.696590    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:30.731721 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:30.731675    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.217976952 podStartE2EDuration="7.73165655s" totalImagesPullingTime="3.493487424s" totalInitContainerRuntime="20.192174ms" isStatefulPod=false podCreationTimestamp="2026-08-05 17:34:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:30.729217055 +0000 UTC m=+78.081891648" watchObservedRunningTime="2026-08-05 17:34:30.73165655 +0000 UTC m=+78.084331145"
Aug 05 17:34:30.862121 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:30.862088    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:30.923571 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:30.923547    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:32.623261 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:32.623218    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:32.623628 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:32.623283    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:32.627968 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:32.627945    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:32.708802 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:32.708780    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:34:32.765152 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:32.765127    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6499c57964-wvz2d"]
Aug 05 17:34:32.888747 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:32.888676    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:32.963840 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:32.963815    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:33.168332 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:33.168257    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:33.268666 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:33.268644    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:34.056330 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:34.056237    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:34:34.073877 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:34.073855    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:34.146694 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:34.146675    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:34.691801 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:34.691775    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-d896f6548-kx6m8"
Aug 05 17:34:34.774594 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:34.774568    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:34.885616 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:34.885586    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:37.188452 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:37.188420    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:37.290528 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:37.290488    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:42.109148 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:42.109116    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:42.109598 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:42.109400    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:34:43.133826 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:43.133794    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:43.197476 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:43.197439    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:45.117466 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:45.117433    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:45.216062 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:45.216031    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:47.305445 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.305367    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f86985667-9gbwx" podUID="f03449d4-b5a5-4908-8119-ee0ec48e10a3" containerName="console" containerID="cri-o://a07227869bd4dcf076f5988c9b123dce8b5796ff31b2a9386462b67a776dc684" gracePeriod=15
Aug 05 17:34:47.563442 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.563376    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:47.574374 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.574336    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f03449d4-b5a5-4908-8119-ee0ec48e10a3-service-ca\") pod \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") "
Aug 05 17:34:47.574529 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.574406    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f03449d4-b5a5-4908-8119-ee0ec48e10a3-oauth-serving-cert\") pod \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") "
Aug 05 17:34:47.574529 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.574425    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f03449d4-b5a5-4908-8119-ee0ec48e10a3-console-serving-cert\") pod \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") "
Aug 05 17:34:47.574529 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.574458    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f03449d4-b5a5-4908-8119-ee0ec48e10a3-console-config\") pod \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") "
Aug 05 17:34:47.574529 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.574484    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qts7r\" (UniqueName: \"kubernetes.io/projected/f03449d4-b5a5-4908-8119-ee0ec48e10a3-kube-api-access-qts7r\") pod \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") "
Aug 05 17:34:47.574529 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.574512    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f03449d4-b5a5-4908-8119-ee0ec48e10a3-console-oauth-config\") pod \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\" (UID: \"f03449d4-b5a5-4908-8119-ee0ec48e10a3\") "
Aug 05 17:34:47.575216 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.574795    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f03449d4-b5a5-4908-8119-ee0ec48e10a3-service-ca" (OuterVolumeSpecName: "service-ca") pod "f03449d4-b5a5-4908-8119-ee0ec48e10a3" (UID: "f03449d4-b5a5-4908-8119-ee0ec48e10a3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:34:47.575216 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.574919    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f03449d4-b5a5-4908-8119-ee0ec48e10a3-console-config" (OuterVolumeSpecName: "console-config") pod "f03449d4-b5a5-4908-8119-ee0ec48e10a3" (UID: "f03449d4-b5a5-4908-8119-ee0ec48e10a3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:34:47.575216 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.574981    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f03449d4-b5a5-4908-8119-ee0ec48e10a3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f03449d4-b5a5-4908-8119-ee0ec48e10a3" (UID: "f03449d4-b5a5-4908-8119-ee0ec48e10a3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:34:47.577976 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.577942    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f03449d4-b5a5-4908-8119-ee0ec48e10a3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f03449d4-b5a5-4908-8119-ee0ec48e10a3" (UID: "f03449d4-b5a5-4908-8119-ee0ec48e10a3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:34:47.578084 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.578054    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f03449d4-b5a5-4908-8119-ee0ec48e10a3-kube-api-access-qts7r" (OuterVolumeSpecName: "kube-api-access-qts7r") pod "f03449d4-b5a5-4908-8119-ee0ec48e10a3" (UID: "f03449d4-b5a5-4908-8119-ee0ec48e10a3"). InnerVolumeSpecName "kube-api-access-qts7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:34:47.578277 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.578242    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f03449d4-b5a5-4908-8119-ee0ec48e10a3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f03449d4-b5a5-4908-8119-ee0ec48e10a3" (UID: "f03449d4-b5a5-4908-8119-ee0ec48e10a3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:34:47.675453 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.675417    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qts7r\" (UniqueName: \"kubernetes.io/projected/f03449d4-b5a5-4908-8119-ee0ec48e10a3-kube-api-access-qts7r\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:34:47.675453 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.675452    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f03449d4-b5a5-4908-8119-ee0ec48e10a3-console-oauth-config\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:34:47.675667 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.675464    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f03449d4-b5a5-4908-8119-ee0ec48e10a3-service-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:34:47.675667 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.675475    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f03449d4-b5a5-4908-8119-ee0ec48e10a3-oauth-serving-cert\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:34:47.675667 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.675487    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f03449d4-b5a5-4908-8119-ee0ec48e10a3-console-serving-cert\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:34:47.675667 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.675497    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f03449d4-b5a5-4908-8119-ee0ec48e10a3-console-config\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:34:47.772310 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.772277    2569 generic.go:356] "Generic (PLEG): container finished" podID="f03449d4-b5a5-4908-8119-ee0ec48e10a3" containerID="a07227869bd4dcf076f5988c9b123dce8b5796ff31b2a9386462b67a776dc684" exitCode=0
Aug 05 17:34:47.772310 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.772321    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f86985667-9gbwx" event={"ID":"f03449d4-b5a5-4908-8119-ee0ec48e10a3","Type":"ContainerDied","Data":"a07227869bd4dcf076f5988c9b123dce8b5796ff31b2a9386462b67a776dc684"}
Aug 05 17:34:47.772578 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.772361    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f86985667-9gbwx" event={"ID":"f03449d4-b5a5-4908-8119-ee0ec48e10a3","Type":"ContainerDied","Data":"c1154c534a8b7e6814fdb0d5092a318ab4169dff5108d9261d6590ffc2c8f9fd"}
Aug 05 17:34:47.772578 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.772381    2569 scope.go:117] "RemoveContainer" containerID="a07227869bd4dcf076f5988c9b123dce8b5796ff31b2a9386462b67a776dc684"
Aug 05 17:34:47.772578 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.772385    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f86985667-9gbwx"
Aug 05 17:34:47.782910 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.782893    2569 scope.go:117] "RemoveContainer" containerID="a07227869bd4dcf076f5988c9b123dce8b5796ff31b2a9386462b67a776dc684"
Aug 05 17:34:47.783265 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:34:47.783156    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a07227869bd4dcf076f5988c9b123dce8b5796ff31b2a9386462b67a776dc684\": container with ID starting with a07227869bd4dcf076f5988c9b123dce8b5796ff31b2a9386462b67a776dc684 not found: ID does not exist" containerID="a07227869bd4dcf076f5988c9b123dce8b5796ff31b2a9386462b67a776dc684"
Aug 05 17:34:47.783265 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.783204    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a07227869bd4dcf076f5988c9b123dce8b5796ff31b2a9386462b67a776dc684"} err="failed to get container status \"a07227869bd4dcf076f5988c9b123dce8b5796ff31b2a9386462b67a776dc684\": rpc error: code = NotFound desc = could not find container \"a07227869bd4dcf076f5988c9b123dce8b5796ff31b2a9386462b67a776dc684\": container with ID starting with a07227869bd4dcf076f5988c9b123dce8b5796ff31b2a9386462b67a776dc684 not found: ID does not exist"
Aug 05 17:34:47.797626 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.797599    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f86985667-9gbwx"]
Aug 05 17:34:47.801919 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:47.801898    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f86985667-9gbwx"]
Aug 05 17:34:49.150104 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:49.150075    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f03449d4-b5a5-4908-8119-ee0ec48e10a3" path="/var/lib/kubelet/pods/f03449d4-b5a5-4908-8119-ee0ec48e10a3/volumes"
Aug 05 17:34:57.674913 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:57.674879    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9vkxl"
Aug 05 17:34:57.784470 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:57.784434    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6499c57964-wvz2d" podUID="b30958f2-4b26-4698-9a9f-bdb8ac268265" containerName="console" containerID="cri-o://2fd1cb14757a9e28bc1dbb15ad246da349acaac01b545e8497036aeaaaa4275f" gracePeriod=15
Aug 05 17:34:58.069435 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.069413    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:58.160842 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.160814    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b30958f2-4b26-4698-9a9f-bdb8ac268265-console-oauth-config\") pod \"b30958f2-4b26-4698-9a9f-bdb8ac268265\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") "
Aug 05 17:34:58.160972 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.160858    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxwss\" (UniqueName: \"kubernetes.io/projected/b30958f2-4b26-4698-9a9f-bdb8ac268265-kube-api-access-gxwss\") pod \"b30958f2-4b26-4698-9a9f-bdb8ac268265\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") "
Aug 05 17:34:58.160972 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.160900    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-trusted-ca-bundle\") pod \"b30958f2-4b26-4698-9a9f-bdb8ac268265\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") "
Aug 05 17:34:58.160972 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.160923    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-console-config\") pod \"b30958f2-4b26-4698-9a9f-bdb8ac268265\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") "
Aug 05 17:34:58.160972 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.160957    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b30958f2-4b26-4698-9a9f-bdb8ac268265-console-serving-cert\") pod \"b30958f2-4b26-4698-9a9f-bdb8ac268265\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") "
Aug 05 17:34:58.161173 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.160992    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-service-ca\") pod \"b30958f2-4b26-4698-9a9f-bdb8ac268265\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") "
Aug 05 17:34:58.161173 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.161025    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-oauth-serving-cert\") pod \"b30958f2-4b26-4698-9a9f-bdb8ac268265\" (UID: \"b30958f2-4b26-4698-9a9f-bdb8ac268265\") "
Aug 05 17:34:58.161408 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.161383    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b30958f2-4b26-4698-9a9f-bdb8ac268265" (UID: "b30958f2-4b26-4698-9a9f-bdb8ac268265"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:34:58.161477 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.161402    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-console-config" (OuterVolumeSpecName: "console-config") pod "b30958f2-4b26-4698-9a9f-bdb8ac268265" (UID: "b30958f2-4b26-4698-9a9f-bdb8ac268265"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:34:58.161477 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.161422    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-service-ca" (OuterVolumeSpecName: "service-ca") pod "b30958f2-4b26-4698-9a9f-bdb8ac268265" (UID: "b30958f2-4b26-4698-9a9f-bdb8ac268265"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:34:58.161831 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.161809    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b30958f2-4b26-4698-9a9f-bdb8ac268265" (UID: "b30958f2-4b26-4698-9a9f-bdb8ac268265"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:34:58.163284 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.163260    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b30958f2-4b26-4698-9a9f-bdb8ac268265-kube-api-access-gxwss" (OuterVolumeSpecName: "kube-api-access-gxwss") pod "b30958f2-4b26-4698-9a9f-bdb8ac268265" (UID: "b30958f2-4b26-4698-9a9f-bdb8ac268265"). InnerVolumeSpecName "kube-api-access-gxwss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:34:58.163376 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.163357    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b30958f2-4b26-4698-9a9f-bdb8ac268265-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b30958f2-4b26-4698-9a9f-bdb8ac268265" (UID: "b30958f2-4b26-4698-9a9f-bdb8ac268265"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:34:58.163692 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.163670    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b30958f2-4b26-4698-9a9f-bdb8ac268265-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b30958f2-4b26-4698-9a9f-bdb8ac268265" (UID: "b30958f2-4b26-4698-9a9f-bdb8ac268265"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:34:58.262123 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.262072    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-service-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:34:58.262123 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.262092    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-oauth-serving-cert\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:34:58.262123 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.262102    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b30958f2-4b26-4698-9a9f-bdb8ac268265-console-oauth-config\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:34:58.262123 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.262112    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxwss\" (UniqueName: \"kubernetes.io/projected/b30958f2-4b26-4698-9a9f-bdb8ac268265-kube-api-access-gxwss\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:34:58.262123 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.262120    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-trusted-ca-bundle\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:34:58.262359 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.262129    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b30958f2-4b26-4698-9a9f-bdb8ac268265-console-config\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:34:58.262359 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.262136    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b30958f2-4b26-4698-9a9f-bdb8ac268265-console-serving-cert\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:34:58.820224 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.820167    2569 generic.go:356] "Generic (PLEG): container finished" podID="b30958f2-4b26-4698-9a9f-bdb8ac268265" containerID="2fd1cb14757a9e28bc1dbb15ad246da349acaac01b545e8497036aeaaaa4275f" exitCode=0
Aug 05 17:34:58.820567 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.820227    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6499c57964-wvz2d" event={"ID":"b30958f2-4b26-4698-9a9f-bdb8ac268265","Type":"ContainerDied","Data":"2fd1cb14757a9e28bc1dbb15ad246da349acaac01b545e8497036aeaaaa4275f"}
Aug 05 17:34:58.820567 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.820232    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6499c57964-wvz2d"
Aug 05 17:34:58.820567 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.820265    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6499c57964-wvz2d" event={"ID":"b30958f2-4b26-4698-9a9f-bdb8ac268265","Type":"ContainerDied","Data":"9c5942d94bd2e0473e420979b1c721942726947238a45b578a1d7e2bf1242158"}
Aug 05 17:34:58.820567 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.820286    2569 scope.go:117] "RemoveContainer" containerID="2fd1cb14757a9e28bc1dbb15ad246da349acaac01b545e8497036aeaaaa4275f"
Aug 05 17:34:58.841762 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.841746    2569 scope.go:117] "RemoveContainer" containerID="2fd1cb14757a9e28bc1dbb15ad246da349acaac01b545e8497036aeaaaa4275f"
Aug 05 17:34:58.842037 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:34:58.842019    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fd1cb14757a9e28bc1dbb15ad246da349acaac01b545e8497036aeaaaa4275f\": container with ID starting with 2fd1cb14757a9e28bc1dbb15ad246da349acaac01b545e8497036aeaaaa4275f not found: ID does not exist" containerID="2fd1cb14757a9e28bc1dbb15ad246da349acaac01b545e8497036aeaaaa4275f"
Aug 05 17:34:58.842087 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.842046    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fd1cb14757a9e28bc1dbb15ad246da349acaac01b545e8497036aeaaaa4275f"} err="failed to get container status \"2fd1cb14757a9e28bc1dbb15ad246da349acaac01b545e8497036aeaaaa4275f\": rpc error: code = NotFound desc = could not find container \"2fd1cb14757a9e28bc1dbb15ad246da349acaac01b545e8497036aeaaaa4275f\": container with ID starting with 2fd1cb14757a9e28bc1dbb15ad246da349acaac01b545e8497036aeaaaa4275f not found: ID does not exist"
Aug 05 17:34:58.844390 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.844368    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6499c57964-wvz2d"]
Aug 05 17:34:58.847424 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:58.847404    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6499c57964-wvz2d"]
Aug 05 17:34:59.149859 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:34:59.149807    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b30958f2-4b26-4698-9a9f-bdb8ac268265" path="/var/lib/kubelet/pods/b30958f2-4b26-4698-9a9f-bdb8ac268265/volumes"
Aug 05 17:35:00.114289 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:00.114261    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:00.197005 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:00.196982    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:02.114343 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:02.114314    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:35:02.118056 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:02.118031    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-798899d854-2rwbz"
Aug 05 17:35:03.391528 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:03.391497    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:03.529964 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:03.529933    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:21.300839 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:21.300801    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:24.055816 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:24.055774    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:24.076585 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:24.076561    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:24.947047 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:24.947021    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:36.789994 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:36.789956    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:36.910960 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:36.910929    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:37.866561 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:37.866534    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 05 17:35:37.866915 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:37.866804    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="alertmanager" containerID="cri-o://e70e7bb5b3b1c5292e566e7ed6b9d63b7fcb4bd3c5e95fa6ba86218d296b558a" gracePeriod=120
Aug 05 17:35:37.866915 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:37.866873    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="kube-rbac-proxy-metric" containerID="cri-o://3863d34bf80e7bba08ce9cbb653f3bd1b3c95bde6f03eb293bf9b6dc2f79c4b0" gracePeriod=120
Aug 05 17:35:37.866915 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:37.866893    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="kube-rbac-proxy-web" containerID="cri-o://7598b1469bb555c351f448b7a98f0cce9a62b5187e27f49f612f61340928a407" gracePeriod=120
Aug 05 17:35:37.867079 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:37.866902    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="config-reloader" containerID="cri-o://004b37dd2cfcb97a4a70a9ea21161ece646d21e6835b540a128861264b7e4be9" gracePeriod=120
Aug 05 17:35:37.867079 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:37.866953    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="prom-label-proxy" containerID="cri-o://c638126fe5b50cb4e506b8cd4a3d5d2b26885bfb89fe1f89d8a6e409e2da3c2f" gracePeriod=120
Aug 05 17:35:37.867079 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:37.866974    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="kube-rbac-proxy" containerID="cri-o://3b3ab342fb9d26aea67f566c4ebfed5a40972a99ef5c88002839a87be2bd51dc" gracePeriod=120
Aug 05 17:35:37.987145 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:37.987120    2569 generic.go:356] "Generic (PLEG): container finished" podID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerID="c638126fe5b50cb4e506b8cd4a3d5d2b26885bfb89fe1f89d8a6e409e2da3c2f" exitCode=0
Aug 05 17:35:37.987145 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:37.987144    2569 generic.go:356] "Generic (PLEG): container finished" podID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerID="3b3ab342fb9d26aea67f566c4ebfed5a40972a99ef5c88002839a87be2bd51dc" exitCode=0
Aug 05 17:35:37.987297 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:37.987153    2569 generic.go:356] "Generic (PLEG): container finished" podID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerID="004b37dd2cfcb97a4a70a9ea21161ece646d21e6835b540a128861264b7e4be9" exitCode=0
Aug 05 17:35:37.987297 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:37.987161    2569 generic.go:356] "Generic (PLEG): container finished" podID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerID="e70e7bb5b3b1c5292e566e7ed6b9d63b7fcb4bd3c5e95fa6ba86218d296b558a" exitCode=0
Aug 05 17:35:37.987297 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:37.987207    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39dfc855-2987-4dab-b0ca-8225ad97d1d9","Type":"ContainerDied","Data":"c638126fe5b50cb4e506b8cd4a3d5d2b26885bfb89fe1f89d8a6e409e2da3c2f"}
Aug 05 17:35:37.987297 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:37.987269    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39dfc855-2987-4dab-b0ca-8225ad97d1d9","Type":"ContainerDied","Data":"3b3ab342fb9d26aea67f566c4ebfed5a40972a99ef5c88002839a87be2bd51dc"}
Aug 05 17:35:37.987297 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:37.987295    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39dfc855-2987-4dab-b0ca-8225ad97d1d9","Type":"ContainerDied","Data":"004b37dd2cfcb97a4a70a9ea21161ece646d21e6835b540a128861264b7e4be9"}
Aug 05 17:35:37.987442 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:37.987321    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39dfc855-2987-4dab-b0ca-8225ad97d1d9","Type":"ContainerDied","Data":"e70e7bb5b3b1c5292e566e7ed6b9d63b7fcb4bd3c5e95fa6ba86218d296b558a"}
Aug 05 17:35:38.995121 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:38.995095    2569 generic.go:356] "Generic (PLEG): container finished" podID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerID="3863d34bf80e7bba08ce9cbb653f3bd1b3c95bde6f03eb293bf9b6dc2f79c4b0" exitCode=0
Aug 05 17:35:38.995121 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:38.995117    2569 generic.go:356] "Generic (PLEG): container finished" podID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerID="7598b1469bb555c351f448b7a98f0cce9a62b5187e27f49f612f61340928a407" exitCode=0
Aug 05 17:35:38.995475 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:38.995166    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39dfc855-2987-4dab-b0ca-8225ad97d1d9","Type":"ContainerDied","Data":"3863d34bf80e7bba08ce9cbb653f3bd1b3c95bde6f03eb293bf9b6dc2f79c4b0"}
Aug 05 17:35:38.995475 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:38.995259    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39dfc855-2987-4dab-b0ca-8225ad97d1d9","Type":"ContainerDied","Data":"7598b1469bb555c351f448b7a98f0cce9a62b5187e27f49f612f61340928a407"}
Aug 05 17:35:39.116980 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.116958    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:39.207213 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.207123    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-web-config\") pod \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") "
Aug 05 17:35:39.207213 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.207155    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39dfc855-2987-4dab-b0ca-8225ad97d1d9-metrics-client-ca\") pod \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") "
Aug 05 17:35:39.207213 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.207198    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") "
Aug 05 17:35:39.207461 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.207236    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-main-tls\") pod \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") "
Aug 05 17:35:39.207461 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.207258    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmgq4\" (UniqueName: \"kubernetes.io/projected/39dfc855-2987-4dab-b0ca-8225ad97d1d9-kube-api-access-hmgq4\") pod \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") "
Aug 05 17:35:39.207461 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.207272    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-config-volume\") pod \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") "
Aug 05 17:35:39.207461 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.207289    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39dfc855-2987-4dab-b0ca-8225ad97d1d9-config-out\") pod \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") "
Aug 05 17:35:39.207461 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.207305    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/39dfc855-2987-4dab-b0ca-8225ad97d1d9-alertmanager-main-db\") pod \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") "
Aug 05 17:35:39.207461 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.207330    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-kube-rbac-proxy-web\") pod \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") "
Aug 05 17:35:39.207461 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.207349    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39dfc855-2987-4dab-b0ca-8225ad97d1d9-alertmanager-trusted-ca-bundle\") pod \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") "
Aug 05 17:35:39.207461 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.207391    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39dfc855-2987-4dab-b0ca-8225ad97d1d9-tls-assets\") pod \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") "
Aug 05 17:35:39.207461 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.207421    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-cluster-tls-config\") pod \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") "
Aug 05 17:35:39.207875 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.207477    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-kube-rbac-proxy\") pod \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\" (UID: \"39dfc855-2987-4dab-b0ca-8225ad97d1d9\") "
Aug 05 17:35:39.207875 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.207537    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39dfc855-2987-4dab-b0ca-8225ad97d1d9-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "39dfc855-2987-4dab-b0ca-8225ad97d1d9" (UID: "39dfc855-2987-4dab-b0ca-8225ad97d1d9"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:35:39.207875 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.207705    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39dfc855-2987-4dab-b0ca-8225ad97d1d9-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "39dfc855-2987-4dab-b0ca-8225ad97d1d9" (UID: "39dfc855-2987-4dab-b0ca-8225ad97d1d9"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:35:39.207875 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.207845    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39dfc855-2987-4dab-b0ca-8225ad97d1d9-metrics-client-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:39.207875 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.207864    2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/39dfc855-2987-4dab-b0ca-8225ad97d1d9-alertmanager-main-db\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:39.208251 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.208169    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39dfc855-2987-4dab-b0ca-8225ad97d1d9-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "39dfc855-2987-4dab-b0ca-8225ad97d1d9" (UID: "39dfc855-2987-4dab-b0ca-8225ad97d1d9"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:35:39.211394 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.211354    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "39dfc855-2987-4dab-b0ca-8225ad97d1d9" (UID: "39dfc855-2987-4dab-b0ca-8225ad97d1d9"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:39.211612 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.211556    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39dfc855-2987-4dab-b0ca-8225ad97d1d9-kube-api-access-hmgq4" (OuterVolumeSpecName: "kube-api-access-hmgq4") pod "39dfc855-2987-4dab-b0ca-8225ad97d1d9" (UID: "39dfc855-2987-4dab-b0ca-8225ad97d1d9"). InnerVolumeSpecName "kube-api-access-hmgq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:35:39.211612 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.211572    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "39dfc855-2987-4dab-b0ca-8225ad97d1d9" (UID: "39dfc855-2987-4dab-b0ca-8225ad97d1d9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:39.211808 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.211775    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-config-volume" (OuterVolumeSpecName: "config-volume") pod "39dfc855-2987-4dab-b0ca-8225ad97d1d9" (UID: "39dfc855-2987-4dab-b0ca-8225ad97d1d9"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:39.211993 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.211967    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "39dfc855-2987-4dab-b0ca-8225ad97d1d9" (UID: "39dfc855-2987-4dab-b0ca-8225ad97d1d9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:39.212119 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.212094    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39dfc855-2987-4dab-b0ca-8225ad97d1d9-config-out" (OuterVolumeSpecName: "config-out") pod "39dfc855-2987-4dab-b0ca-8225ad97d1d9" (UID: "39dfc855-2987-4dab-b0ca-8225ad97d1d9"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:35:39.212316 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.212293    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39dfc855-2987-4dab-b0ca-8225ad97d1d9-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "39dfc855-2987-4dab-b0ca-8225ad97d1d9" (UID: "39dfc855-2987-4dab-b0ca-8225ad97d1d9"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:35:39.212654 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.212628    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "39dfc855-2987-4dab-b0ca-8225ad97d1d9" (UID: "39dfc855-2987-4dab-b0ca-8225ad97d1d9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:39.215598 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.215568    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "39dfc855-2987-4dab-b0ca-8225ad97d1d9" (UID: "39dfc855-2987-4dab-b0ca-8225ad97d1d9"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:39.222066 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.222046    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-web-config" (OuterVolumeSpecName: "web-config") pod "39dfc855-2987-4dab-b0ca-8225ad97d1d9" (UID: "39dfc855-2987-4dab-b0ca-8225ad97d1d9"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:39.308552 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.308522    2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39dfc855-2987-4dab-b0ca-8225ad97d1d9-tls-assets\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:39.308552 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.308553    2569 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-cluster-tls-config\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:39.308552 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.308562    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:39.308703 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.308571    2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-web-config\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:39.308703 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.308580    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:39.308703 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.308589    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-main-tls\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:39.308703 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.308598    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmgq4\" (UniqueName: \"kubernetes.io/projected/39dfc855-2987-4dab-b0ca-8225ad97d1d9-kube-api-access-hmgq4\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:39.308703 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.308606    2569 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:39.308703 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.308615    2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39dfc855-2987-4dab-b0ca-8225ad97d1d9-config-out\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:39.308703 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.308622    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39dfc855-2987-4dab-b0ca-8225ad97d1d9-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:39.308703 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:39.308630    2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39dfc855-2987-4dab-b0ca-8225ad97d1d9-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:40.002367 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.002339    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.002760 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.002341    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39dfc855-2987-4dab-b0ca-8225ad97d1d9","Type":"ContainerDied","Data":"fa4e736bd303a3f97f36c7a5f506e5a994aaca1bca17c0fb122476abee894b2d"}
Aug 05 17:35:40.002760 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.002487    2569 scope.go:117] "RemoveContainer" containerID="c638126fe5b50cb4e506b8cd4a3d5d2b26885bfb89fe1f89d8a6e409e2da3c2f"
Aug 05 17:35:40.010025 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.009943    2569 scope.go:117] "RemoveContainer" containerID="3863d34bf80e7bba08ce9cbb653f3bd1b3c95bde6f03eb293bf9b6dc2f79c4b0"
Aug 05 17:35:40.020880 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.020859    2569 scope.go:117] "RemoveContainer" containerID="3b3ab342fb9d26aea67f566c4ebfed5a40972a99ef5c88002839a87be2bd51dc"
Aug 05 17:35:40.027776 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.027756    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 05 17:35:40.028058 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.028044    2569 scope.go:117] "RemoveContainer" containerID="7598b1469bb555c351f448b7a98f0cce9a62b5187e27f49f612f61340928a407"
Aug 05 17:35:40.033114 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.033081    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 05 17:35:40.035310 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.035283    2569 scope.go:117] "RemoveContainer" containerID="004b37dd2cfcb97a4a70a9ea21161ece646d21e6835b540a128861264b7e4be9"
Aug 05 17:35:40.042043 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.042028    2569 scope.go:117] "RemoveContainer" containerID="e70e7bb5b3b1c5292e566e7ed6b9d63b7fcb4bd3c5e95fa6ba86218d296b558a"
Aug 05 17:35:40.048760 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.048745    2569 scope.go:117] "RemoveContainer" containerID="aed68aabdd6955c9123d91b756a4dbc145bf488c44f804199d365850267414b9"
Aug 05 17:35:40.059093 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059073    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 05 17:35:40.059477 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059456    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="init-config-reloader"
Aug 05 17:35:40.059477 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059474    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="init-config-reloader"
Aug 05 17:35:40.059588 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059485    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="config-reloader"
Aug 05 17:35:40.059588 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059491    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="config-reloader"
Aug 05 17:35:40.059588 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059501    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="kube-rbac-proxy-web"
Aug 05 17:35:40.059588 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059507    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="kube-rbac-proxy-web"
Aug 05 17:35:40.059588 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059515    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f03449d4-b5a5-4908-8119-ee0ec48e10a3" containerName="console"
Aug 05 17:35:40.059588 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059520    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f03449d4-b5a5-4908-8119-ee0ec48e10a3" containerName="console"
Aug 05 17:35:40.059588 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059527    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="kube-rbac-proxy"
Aug 05 17:35:40.059588 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059532    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="kube-rbac-proxy"
Aug 05 17:35:40.059588 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059537    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="prom-label-proxy"
Aug 05 17:35:40.059588 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059542    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="prom-label-proxy"
Aug 05 17:35:40.059588 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059547    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b30958f2-4b26-4698-9a9f-bdb8ac268265" containerName="console"
Aug 05 17:35:40.059588 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059552    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b30958f2-4b26-4698-9a9f-bdb8ac268265" containerName="console"
Aug 05 17:35:40.059588 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059560    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="alertmanager"
Aug 05 17:35:40.059588 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059565    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="alertmanager"
Aug 05 17:35:40.059588 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059576    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="kube-rbac-proxy-metric"
Aug 05 17:35:40.059588 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059581    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="kube-rbac-proxy-metric"
Aug 05 17:35:40.060016 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059625    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="prom-label-proxy"
Aug 05 17:35:40.060016 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059632    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="kube-rbac-proxy-web"
Aug 05 17:35:40.060016 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059638    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f03449d4-b5a5-4908-8119-ee0ec48e10a3" containerName="console"
Aug 05 17:35:40.060016 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059643    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="alertmanager"
Aug 05 17:35:40.060016 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059648    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="kube-rbac-proxy-metric"
Aug 05 17:35:40.060016 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059656    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="kube-rbac-proxy"
Aug 05 17:35:40.060016 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059662    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b30958f2-4b26-4698-9a9f-bdb8ac268265" containerName="console"
Aug 05 17:35:40.060016 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.059666    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" containerName="config-reloader"
Aug 05 17:35:40.063072 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.063055    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.066428 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.066111    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jcwdk\""
Aug 05 17:35:40.066517 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.066424    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Aug 05 17:35:40.066678 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.066655    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Aug 05 17:35:40.066754 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.066692    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Aug 05 17:35:40.066754 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.066709    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Aug 05 17:35:40.066859 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.066715    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Aug 05 17:35:40.066859 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.066726    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Aug 05 17:35:40.066859 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.066845    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Aug 05 17:35:40.067168 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.067144    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Aug 05 17:35:40.071169 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.071148    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Aug 05 17:35:40.077910 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.077890    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 05 17:35:40.113976 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.113952    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.114067 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.113988    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.114067 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.114007    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-config-out\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.114067 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.114032    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.114163 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.114075    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.114163 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.114098    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.114163 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.114118    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz9d8\" (UniqueName: \"kubernetes.io/projected/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-kube-api-access-vz9d8\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.114163 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.114148    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.114306 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.114167    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-web-config\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.114306 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.114232    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.114306 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.114274    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-config-volume\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.114306 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.114295    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.114418 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.114322    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.215508 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.215448    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.215508 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.215490    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.215619 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.215536    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.215619 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.215561    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.215619 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.215587    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-config-out\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.215770 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.215644    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.215823 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.215791    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.215871 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.215834    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.215924 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.215868    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz9d8\" (UniqueName: \"kubernetes.io/projected/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-kube-api-access-vz9d8\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.215977 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.215929    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.215977 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.215960    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-web-config\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.216081 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.216000    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.216081 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.216018    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.216081 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.216050    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-config-volume\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.216487 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.216419    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.217487 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.216997    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.218995 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.218624    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.218995 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.218745    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.218995 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.218856    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.218995 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.218958    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-config-out\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.219240 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.219088    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.219664 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.219642    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.219725 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.219647    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.220019 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.220000    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-web-config\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.220840 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.220822    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-config-volume\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.224843 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.224822    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz9d8\" (UniqueName: \"kubernetes.io/projected/3ae4e325-445d-4cf5-acc1-40ccd5bff3ed-kube-api-access-vz9d8\") pod \"alertmanager-main-0\" (UID: \"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.374555 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.374529    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:35:40.510342 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:40.510321    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 05 17:35:40.512636 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:35:40.512615    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ae4e325_445d_4cf5_acc1_40ccd5bff3ed.slice/crio-8ff46f824db542bfd01bd8655af755624d437bad4e00f03f3af9bac742da8003 WatchSource:0}: Error finding container 8ff46f824db542bfd01bd8655af755624d437bad4e00f03f3af9bac742da8003: Status 404 returned error can't find the container with id 8ff46f824db542bfd01bd8655af755624d437bad4e00f03f3af9bac742da8003
Aug 05 17:35:41.007581 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:41.007552    2569 generic.go:356] "Generic (PLEG): container finished" podID="3ae4e325-445d-4cf5-acc1-40ccd5bff3ed" containerID="7bd513cc7351a4774d2de580edd8f77e96b7c1bc95e5bdea7854bb68032e7239" exitCode=0
Aug 05 17:35:41.007956 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:41.007653    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed","Type":"ContainerDied","Data":"7bd513cc7351a4774d2de580edd8f77e96b7c1bc95e5bdea7854bb68032e7239"}
Aug 05 17:35:41.007956 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:41.007742    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed","Type":"ContainerStarted","Data":"8ff46f824db542bfd01bd8655af755624d437bad4e00f03f3af9bac742da8003"}
Aug 05 17:35:41.150481 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:41.150360    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39dfc855-2987-4dab-b0ca-8225ad97d1d9" path="/var/lib/kubelet/pods/39dfc855-2987-4dab-b0ca-8225ad97d1d9/volumes"
Aug 05 17:35:42.016313 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.016277    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed","Type":"ContainerStarted","Data":"73c160ea65382cc702d2ad5187569e1a520fb6239097fa33242da395cf40bb62"}
Aug 05 17:35:42.016794 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.016339    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed","Type":"ContainerStarted","Data":"b831a8d0276770faa35853a283802b89a0dc8f1cad960ada1e9b1495f3a6d1fa"}
Aug 05 17:35:42.016794 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.016365    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed","Type":"ContainerStarted","Data":"5112e4bed5d152fc61c334fdb60bde46b0e4d5d1fd94d2b4e7beb50c6e981ff5"}
Aug 05 17:35:42.016794 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.016388    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed","Type":"ContainerStarted","Data":"debe2b774ce7edd7dd83e2b07c8dfc91fec7018a67f50e4025ea2aefd3c43ec5"}
Aug 05 17:35:42.016794 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.016414    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed","Type":"ContainerStarted","Data":"6ff418e3401a375b301d07648d631e16bf79a18171fdb6cc609ec59cb9eebfba"}
Aug 05 17:35:42.016794 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.016436    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ae4e325-445d-4cf5-acc1-40ccd5bff3ed","Type":"ContainerStarted","Data":"9eb98e2b4eecd26fba1569c3d64722ba00ac895a5b0a39cf964440cc8b785757"}
Aug 05 17:35:42.062451 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.062402    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.046486127 podStartE2EDuration="2.062385351s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.899224ms" isStatefulPod=false podCreationTimestamp="2026-08-05 17:35:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:35:42.059663142 +0000 UTC m=+149.412337738" watchObservedRunningTime="2026-08-05 17:35:42.062385351 +0000 UTC m=+149.415059945"
Aug 05 17:35:42.246958 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.246924    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 05 17:35:42.247328 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.247283    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="prometheus" containerID="cri-o://04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb" gracePeriod=600
Aug 05 17:35:42.247440 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.247313    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="kube-rbac-proxy" containerID="cri-o://6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518" gracePeriod=600
Aug 05 17:35:42.247440 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.247313    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="thanos-sidecar" containerID="cri-o://5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad" gracePeriod=600
Aug 05 17:35:42.247440 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.247352    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="kube-rbac-proxy-web" containerID="cri-o://e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df" gracePeriod=600
Aug 05 17:35:42.247440 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.247368    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="config-reloader" containerID="cri-o://5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1" gracePeriod=600
Aug 05 17:35:42.247694 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.247364    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="kube-rbac-proxy-thanos" containerID="cri-o://63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31" gracePeriod=600
Aug 05 17:35:42.491523 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.491447    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:42.535624 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.535563    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.535624 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.535616    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-prometheus-k8s-tls\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.535833 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.535635    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqcc2\" (UniqueName: \"kubernetes.io/projected/132dda5b-3438-4121-90d5-b6f71da27477-kube-api-access-qqcc2\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.535833 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.535656    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-grpc-tls\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.535833 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.535789    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-db\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.535833 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.535822    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-rulefiles-1\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.536072 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.535846    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-configmap-metrics-client-ca\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.536072 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.535880    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-config\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.536072 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.535922    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/132dda5b-3438-4121-90d5-b6f71da27477-config-out\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.536072 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.535957    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-web-config\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.536072 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.536015    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.536072 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.536043    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-trusted-ca-bundle\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.536541 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.536076    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/132dda5b-3438-4121-90d5-b6f71da27477-tls-assets\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.536541 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.536105    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-kube-rbac-proxy\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.536541 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.536132    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-configmap-serving-certs-ca-bundle\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.536541 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.536166    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-rulefiles-0\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.536541 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.536236    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-rulefiles-2\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.536541 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.536263    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-configmap-kubelet-serving-ca-bundle\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.536541 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.536295    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-metrics-client-certs\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.536541 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.536307    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:35:42.536541 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.536322    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-thanos-prometheus-http-client-file\") pod \"132dda5b-3438-4121-90d5-b6f71da27477\" (UID: \"132dda5b-3438-4121-90d5-b6f71da27477\") "
Aug 05 17:35:42.536541 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.536535    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:35:42.537090 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.536641    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:42.537090 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.536661    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-configmap-metrics-client-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:42.537249 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.537220    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:35:42.537440 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.537012    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:35:42.538374 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.538342    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:35:42.538627 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.538602    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:35:42.539032 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.539006    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:35:42.539872 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.539824    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:35:42.543512 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.541588    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-config" (OuterVolumeSpecName: "config") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:42.543512 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.542056    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/132dda5b-3438-4121-90d5-b6f71da27477-kube-api-access-qqcc2" (OuterVolumeSpecName: "kube-api-access-qqcc2") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "kube-api-access-qqcc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:35:42.543512 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.542220    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:42.543512 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.542349    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:42.543512 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.542503    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:42.543512 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.542722    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:42.543512 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.542768    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:42.543512 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.543447    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:42.543512 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.543461    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/132dda5b-3438-4121-90d5-b6f71da27477-config-out" (OuterVolumeSpecName: "config-out") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:35:42.544724 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.544689    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:42.545286 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.545151    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/132dda5b-3438-4121-90d5-b6f71da27477-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:35:42.557512 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.557478    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-web-config" (OuterVolumeSpecName: "web-config") pod "132dda5b-3438-4121-90d5-b6f71da27477" (UID: "132dda5b-3438-4121-90d5-b6f71da27477"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:42.638000 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.637979    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:42.638000 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.638001    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:42.638148 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.638010    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqcc2\" (UniqueName: \"kubernetes.io/projected/132dda5b-3438-4121-90d5-b6f71da27477-kube-api-access-qqcc2\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:42.638148 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.638020    2569 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-grpc-tls\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:42.638148 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.638029    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-db\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:42.638148 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.638037    2569 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-config\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:42.638148 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.638049    2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/132dda5b-3438-4121-90d5-b6f71da27477-config-out\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:42.638148 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.638060    2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-web-config\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:42.638148 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.638068    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:42.638148 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.638075    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:42.638148 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.638083    2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/132dda5b-3438-4121-90d5-b6f71da27477-tls-assets\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:42.638148 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.638090    2569 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-kube-rbac-proxy\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:42.638148 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.638099    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:42.638148 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.638108    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:42.638148 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.638116    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:42.638148 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.638131    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/132dda5b-3438-4121-90d5-b6f71da27477-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:42.638148 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.638144    2569 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-secret-metrics-client-certs\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:42.638148 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:42.638152    2569 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/132dda5b-3438-4121-90d5-b6f71da27477-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:35:43.023156 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.023127    2569 generic.go:356] "Generic (PLEG): container finished" podID="132dda5b-3438-4121-90d5-b6f71da27477" containerID="63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31" exitCode=0
Aug 05 17:35:43.023156 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.023148    2569 generic.go:356] "Generic (PLEG): container finished" podID="132dda5b-3438-4121-90d5-b6f71da27477" containerID="6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518" exitCode=0
Aug 05 17:35:43.023156 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.023154    2569 generic.go:356] "Generic (PLEG): container finished" podID="132dda5b-3438-4121-90d5-b6f71da27477" containerID="e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df" exitCode=0
Aug 05 17:35:43.023156 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.023159    2569 generic.go:356] "Generic (PLEG): container finished" podID="132dda5b-3438-4121-90d5-b6f71da27477" containerID="5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad" exitCode=0
Aug 05 17:35:43.023566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.023164    2569 generic.go:356] "Generic (PLEG): container finished" podID="132dda5b-3438-4121-90d5-b6f71da27477" containerID="5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1" exitCode=0
Aug 05 17:35:43.023566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.023170    2569 generic.go:356] "Generic (PLEG): container finished" podID="132dda5b-3438-4121-90d5-b6f71da27477" containerID="04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb" exitCode=0
Aug 05 17:35:43.023566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.023227    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.023566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.023218    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"132dda5b-3438-4121-90d5-b6f71da27477","Type":"ContainerDied","Data":"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31"}
Aug 05 17:35:43.023566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.023385    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"132dda5b-3438-4121-90d5-b6f71da27477","Type":"ContainerDied","Data":"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518"}
Aug 05 17:35:43.023566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.023419    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"132dda5b-3438-4121-90d5-b6f71da27477","Type":"ContainerDied","Data":"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df"}
Aug 05 17:35:43.023566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.023444    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"132dda5b-3438-4121-90d5-b6f71da27477","Type":"ContainerDied","Data":"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad"}
Aug 05 17:35:43.023566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.023473    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"132dda5b-3438-4121-90d5-b6f71da27477","Type":"ContainerDied","Data":"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1"}
Aug 05 17:35:43.023566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.023497    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"132dda5b-3438-4121-90d5-b6f71da27477","Type":"ContainerDied","Data":"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb"}
Aug 05 17:35:43.023566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.023445    2569 scope.go:117] "RemoveContainer" containerID="63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31"
Aug 05 17:35:43.023566 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.023520    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"132dda5b-3438-4121-90d5-b6f71da27477","Type":"ContainerDied","Data":"a7be02f7ce84f5674498ed1ce74237e2692ffb206aa213ad59365925bb336a76"}
Aug 05 17:35:43.032171 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.032155    2569 scope.go:117] "RemoveContainer" containerID="6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518"
Aug 05 17:35:43.039987 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.039971    2569 scope.go:117] "RemoveContainer" containerID="e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df"
Aug 05 17:35:43.046627 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.046613    2569 scope.go:117] "RemoveContainer" containerID="5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad"
Aug 05 17:35:43.049553 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.049532    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 05 17:35:43.053655 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.053639    2569 scope.go:117] "RemoveContainer" containerID="5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1"
Aug 05 17:35:43.060632 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.060613    2569 scope.go:117] "RemoveContainer" containerID="04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb"
Aug 05 17:35:43.061658 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.061642    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 05 17:35:43.067531 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.067517    2569 scope.go:117] "RemoveContainer" containerID="02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048"
Aug 05 17:35:43.073818 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.073805    2569 scope.go:117] "RemoveContainer" containerID="63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31"
Aug 05 17:35:43.074034 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:35:43.074009    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31\": container with ID starting with 63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31 not found: ID does not exist" containerID="63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31"
Aug 05 17:35:43.074116 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.074046    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31"} err="failed to get container status \"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31\": rpc error: code = NotFound desc = could not find container \"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31\": container with ID starting with 63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31 not found: ID does not exist"
Aug 05 17:35:43.074116 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.074063    2569 scope.go:117] "RemoveContainer" containerID="6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518"
Aug 05 17:35:43.074358 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:35:43.074340    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518\": container with ID starting with 6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518 not found: ID does not exist" containerID="6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518"
Aug 05 17:35:43.074396 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.074366    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518"} err="failed to get container status \"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518\": rpc error: code = NotFound desc = could not find container \"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518\": container with ID starting with 6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518 not found: ID does not exist"
Aug 05 17:35:43.074396 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.074390    2569 scope.go:117] "RemoveContainer" containerID="e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df"
Aug 05 17:35:43.074606 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:35:43.074590    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df\": container with ID starting with e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df not found: ID does not exist" containerID="e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df"
Aug 05 17:35:43.074657 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.074610    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df"} err="failed to get container status \"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df\": rpc error: code = NotFound desc = could not find container \"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df\": container with ID starting with e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df not found: ID does not exist"
Aug 05 17:35:43.074657 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.074623    2569 scope.go:117] "RemoveContainer" containerID="5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad"
Aug 05 17:35:43.074829 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:35:43.074812    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad\": container with ID starting with 5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad not found: ID does not exist" containerID="5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad"
Aug 05 17:35:43.074873 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.074833    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad"} err="failed to get container status \"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad\": rpc error: code = NotFound desc = could not find container \"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad\": container with ID starting with 5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad not found: ID does not exist"
Aug 05 17:35:43.074873 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.074848    2569 scope.go:117] "RemoveContainer" containerID="5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1"
Aug 05 17:35:43.075035 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:35:43.075022    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1\": container with ID starting with 5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1 not found: ID does not exist" containerID="5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1"
Aug 05 17:35:43.075071 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.075039    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1"} err="failed to get container status \"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1\": rpc error: code = NotFound desc = could not find container \"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1\": container with ID starting with 5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1 not found: ID does not exist"
Aug 05 17:35:43.075071 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.075052    2569 scope.go:117] "RemoveContainer" containerID="04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb"
Aug 05 17:35:43.075366 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:35:43.075350    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb\": container with ID starting with 04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb not found: ID does not exist" containerID="04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb"
Aug 05 17:35:43.075430 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.075370    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb"} err="failed to get container status \"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb\": rpc error: code = NotFound desc = could not find container \"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb\": container with ID starting with 04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb not found: ID does not exist"
Aug 05 17:35:43.075430 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.075382    2569 scope.go:117] "RemoveContainer" containerID="02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048"
Aug 05 17:35:43.075607 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:35:43.075591    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048\": container with ID starting with 02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048 not found: ID does not exist" containerID="02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048"
Aug 05 17:35:43.075670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.075622    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048"} err="failed to get container status \"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048\": rpc error: code = NotFound desc = could not find container \"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048\": container with ID starting with 02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048 not found: ID does not exist"
Aug 05 17:35:43.075670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.075643    2569 scope.go:117] "RemoveContainer" containerID="63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31"
Aug 05 17:35:43.075863 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.075841    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31"} err="failed to get container status \"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31\": rpc error: code = NotFound desc = could not find container \"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31\": container with ID starting with 63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31 not found: ID does not exist"
Aug 05 17:35:43.075921 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.075865    2569 scope.go:117] "RemoveContainer" containerID="6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518"
Aug 05 17:35:43.076068 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.076052    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518"} err="failed to get container status \"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518\": rpc error: code = NotFound desc = could not find container \"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518\": container with ID starting with 6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518 not found: ID does not exist"
Aug 05 17:35:43.076113 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.076068    2569 scope.go:117] "RemoveContainer" containerID="e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df"
Aug 05 17:35:43.076340 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.076322    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df"} err="failed to get container status \"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df\": rpc error: code = NotFound desc = could not find container \"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df\": container with ID starting with e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df not found: ID does not exist"
Aug 05 17:35:43.076413 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.076342    2569 scope.go:117] "RemoveContainer" containerID="5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad"
Aug 05 17:35:43.076603 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.076586    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad"} err="failed to get container status \"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad\": rpc error: code = NotFound desc = could not find container \"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad\": container with ID starting with 5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad not found: ID does not exist"
Aug 05 17:35:43.076651 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.076604    2569 scope.go:117] "RemoveContainer" containerID="5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1"
Aug 05 17:35:43.076827 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.076810    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1"} err="failed to get container status \"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1\": rpc error: code = NotFound desc = could not find container \"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1\": container with ID starting with 5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1 not found: ID does not exist"
Aug 05 17:35:43.076866 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.076829    2569 scope.go:117] "RemoveContainer" containerID="04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb"
Aug 05 17:35:43.077056 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.077036    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb"} err="failed to get container status \"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb\": rpc error: code = NotFound desc = could not find container \"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb\": container with ID starting with 04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb not found: ID does not exist"
Aug 05 17:35:43.077131 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.077058    2569 scope.go:117] "RemoveContainer" containerID="02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048"
Aug 05 17:35:43.077310 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.077292    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048"} err="failed to get container status \"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048\": rpc error: code = NotFound desc = could not find container \"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048\": container with ID starting with 02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048 not found: ID does not exist"
Aug 05 17:35:43.077354 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.077310    2569 scope.go:117] "RemoveContainer" containerID="63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31"
Aug 05 17:35:43.077538 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.077522    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31"} err="failed to get container status \"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31\": rpc error: code = NotFound desc = could not find container \"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31\": container with ID starting with 63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31 not found: ID does not exist"
Aug 05 17:35:43.077578 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.077540    2569 scope.go:117] "RemoveContainer" containerID="6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518"
Aug 05 17:35:43.077739 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.077722    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518"} err="failed to get container status \"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518\": rpc error: code = NotFound desc = could not find container \"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518\": container with ID starting with 6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518 not found: ID does not exist"
Aug 05 17:35:43.077810 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.077740    2569 scope.go:117] "RemoveContainer" containerID="e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df"
Aug 05 17:35:43.077951 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.077934    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df"} err="failed to get container status \"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df\": rpc error: code = NotFound desc = could not find container \"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df\": container with ID starting with e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df not found: ID does not exist"
Aug 05 17:35:43.077994 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.077952    2569 scope.go:117] "RemoveContainer" containerID="5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad"
Aug 05 17:35:43.078127 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.078108    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad"} err="failed to get container status \"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad\": rpc error: code = NotFound desc = could not find container \"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad\": container with ID starting with 5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad not found: ID does not exist"
Aug 05 17:35:43.078196 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.078129    2569 scope.go:117] "RemoveContainer" containerID="5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1"
Aug 05 17:35:43.078361 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.078345    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1"} err="failed to get container status \"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1\": rpc error: code = NotFound desc = could not find container \"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1\": container with ID starting with 5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1 not found: ID does not exist"
Aug 05 17:35:43.078406 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.078362    2569 scope.go:117] "RemoveContainer" containerID="04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb"
Aug 05 17:35:43.078539 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.078524    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb"} err="failed to get container status \"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb\": rpc error: code = NotFound desc = could not find container \"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb\": container with ID starting with 04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb not found: ID does not exist"
Aug 05 17:35:43.078580 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.078539    2569 scope.go:117] "RemoveContainer" containerID="02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048"
Aug 05 17:35:43.078740 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.078725    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048"} err="failed to get container status \"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048\": rpc error: code = NotFound desc = could not find container \"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048\": container with ID starting with 02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048 not found: ID does not exist"
Aug 05 17:35:43.078791 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.078739    2569 scope.go:117] "RemoveContainer" containerID="63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31"
Aug 05 17:35:43.078923 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.078903    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31"} err="failed to get container status \"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31\": rpc error: code = NotFound desc = could not find container \"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31\": container with ID starting with 63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31 not found: ID does not exist"
Aug 05 17:35:43.078964 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.078923    2569 scope.go:117] "RemoveContainer" containerID="6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518"
Aug 05 17:35:43.079103 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.079089    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518"} err="failed to get container status \"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518\": rpc error: code = NotFound desc = could not find container \"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518\": container with ID starting with 6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518 not found: ID does not exist"
Aug 05 17:35:43.079163 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.079103    2569 scope.go:117] "RemoveContainer" containerID="e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df"
Aug 05 17:35:43.079300 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.079285    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df"} err="failed to get container status \"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df\": rpc error: code = NotFound desc = could not find container \"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df\": container with ID starting with e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df not found: ID does not exist"
Aug 05 17:35:43.079340 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.079300    2569 scope.go:117] "RemoveContainer" containerID="5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad"
Aug 05 17:35:43.079483 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.079469    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad"} err="failed to get container status \"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad\": rpc error: code = NotFound desc = could not find container \"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad\": container with ID starting with 5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad not found: ID does not exist"
Aug 05 17:35:43.079537 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.079483    2569 scope.go:117] "RemoveContainer" containerID="5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1"
Aug 05 17:35:43.079675 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.079658    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1"} err="failed to get container status \"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1\": rpc error: code = NotFound desc = could not find container \"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1\": container with ID starting with 5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1 not found: ID does not exist"
Aug 05 17:35:43.079727 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.079675    2569 scope.go:117] "RemoveContainer" containerID="04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb"
Aug 05 17:35:43.079864 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.079846    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb"} err="failed to get container status \"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb\": rpc error: code = NotFound desc = could not find container \"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb\": container with ID starting with 04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb not found: ID does not exist"
Aug 05 17:35:43.079908 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.079866    2569 scope.go:117] "RemoveContainer" containerID="02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048"
Aug 05 17:35:43.081680 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.081663    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048"} err="failed to get container status \"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048\": rpc error: code = NotFound desc = could not find container \"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048\": container with ID starting with 02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048 not found: ID does not exist"
Aug 05 17:35:43.081680 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.081679    2569 scope.go:117] "RemoveContainer" containerID="63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31"
Aug 05 17:35:43.081907 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.081888    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31"} err="failed to get container status \"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31\": rpc error: code = NotFound desc = could not find container \"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31\": container with ID starting with 63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31 not found: ID does not exist"
Aug 05 17:35:43.081982 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.081912    2569 scope.go:117] "RemoveContainer" containerID="6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518"
Aug 05 17:35:43.082094 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.082078    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518"} err="failed to get container status \"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518\": rpc error: code = NotFound desc = could not find container \"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518\": container with ID starting with 6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518 not found: ID does not exist"
Aug 05 17:35:43.082146 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.082095    2569 scope.go:117] "RemoveContainer" containerID="e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df"
Aug 05 17:35:43.082390 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.082371    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df"} err="failed to get container status \"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df\": rpc error: code = NotFound desc = could not find container \"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df\": container with ID starting with e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df not found: ID does not exist"
Aug 05 17:35:43.082453 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.082390    2569 scope.go:117] "RemoveContainer" containerID="5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad"
Aug 05 17:35:43.082646 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.082621    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad"} err="failed to get container status \"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad\": rpc error: code = NotFound desc = could not find container \"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad\": container with ID starting with 5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad not found: ID does not exist"
Aug 05 17:35:43.082693 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.082648    2569 scope.go:117] "RemoveContainer" containerID="5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1"
Aug 05 17:35:43.082919 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.082902    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1"} err="failed to get container status \"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1\": rpc error: code = NotFound desc = could not find container \"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1\": container with ID starting with 5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1 not found: ID does not exist"
Aug 05 17:35:43.082985 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.082920    2569 scope.go:117] "RemoveContainer" containerID="04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb"
Aug 05 17:35:43.083171 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.083151    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb"} err="failed to get container status \"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb\": rpc error: code = NotFound desc = could not find container \"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb\": container with ID starting with 04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb not found: ID does not exist"
Aug 05 17:35:43.083253 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.083189    2569 scope.go:117] "RemoveContainer" containerID="02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048"
Aug 05 17:35:43.083429 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.083401    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048"} err="failed to get container status \"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048\": rpc error: code = NotFound desc = could not find container \"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048\": container with ID starting with 02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048 not found: ID does not exist"
Aug 05 17:35:43.083429 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.083427    2569 scope.go:117] "RemoveContainer" containerID="63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31"
Aug 05 17:35:43.083681 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.083663    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31"} err="failed to get container status \"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31\": rpc error: code = NotFound desc = could not find container \"63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31\": container with ID starting with 63b62c586cd75e976ff608d4b97c6068486d7e3e6b94b20730511c412cdfdb31 not found: ID does not exist"
Aug 05 17:35:43.083730 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.083683    2569 scope.go:117] "RemoveContainer" containerID="6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518"
Aug 05 17:35:43.083897 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.083880    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518"} err="failed to get container status \"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518\": rpc error: code = NotFound desc = could not find container \"6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518\": container with ID starting with 6d93f7a8563739fece4081391c0425ec6bc86bc890304b4bfcea200a5856e518 not found: ID does not exist"
Aug 05 17:35:43.083962 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.083899    2569 scope.go:117] "RemoveContainer" containerID="e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df"
Aug 05 17:35:43.084113 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.084093    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df"} err="failed to get container status \"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df\": rpc error: code = NotFound desc = could not find container \"e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df\": container with ID starting with e34af93ec72180b12ccf2cb1679f2478d34677a412e41edc7af84a036d4d89df not found: ID does not exist"
Aug 05 17:35:43.084113 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.084113    2569 scope.go:117] "RemoveContainer" containerID="5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad"
Aug 05 17:35:43.084382 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.084364    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad"} err="failed to get container status \"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad\": rpc error: code = NotFound desc = could not find container \"5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad\": container with ID starting with 5f91a638d3ad530126ddc82f79ece19eb133eab3be94d414b50169a3d6f769ad not found: ID does not exist"
Aug 05 17:35:43.084454 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.084384    2569 scope.go:117] "RemoveContainer" containerID="5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1"
Aug 05 17:35:43.084595 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.084573    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1"} err="failed to get container status \"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1\": rpc error: code = NotFound desc = could not find container \"5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1\": container with ID starting with 5a8fe6bb9dc8ddcef9955e3d5b6060037d8131960a10a8a4108c7e74a12acfb1 not found: ID does not exist"
Aug 05 17:35:43.084691 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.084596    2569 scope.go:117] "RemoveContainer" containerID="04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb"
Aug 05 17:35:43.084778 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.084760    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb"} err="failed to get container status \"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb\": rpc error: code = NotFound desc = could not find container \"04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb\": container with ID starting with 04ccaef885c2a35907083b909160264ed974354f3c8910d8c05e9307061e0feb not found: ID does not exist"
Aug 05 17:35:43.084826 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.084781    2569 scope.go:117] "RemoveContainer" containerID="02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048"
Aug 05 17:35:43.085093 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.085075    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048"} err="failed to get container status \"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048\": rpc error: code = NotFound desc = could not find container \"02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048\": container with ID starting with 02f7f235be6bf3c83fe7c6274c8e609f324a4299fcafb99921d22b9a97137048 not found: ID does not exist"
Aug 05 17:35:43.102029 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102007    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102423    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="config-reloader"
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102442    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="config-reloader"
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102458    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="kube-rbac-proxy-web"
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102468    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="kube-rbac-proxy-web"
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102487    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="kube-rbac-proxy-thanos"
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102495    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="kube-rbac-proxy-thanos"
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102508    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="prometheus"
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102516    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="prometheus"
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102529    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="init-config-reloader"
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102538    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="init-config-reloader"
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102561    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="kube-rbac-proxy"
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102568    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="kube-rbac-proxy"
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102579    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="thanos-sidecar"
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102588    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="thanos-sidecar"
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102653    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="kube-rbac-proxy-web"
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102665    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="prometheus"
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102673    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="thanos-sidecar"
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102681    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="kube-rbac-proxy-thanos"
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102690    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="config-reloader"
Aug 05 17:35:43.104166 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.102698    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="132dda5b-3438-4121-90d5-b6f71da27477" containerName="kube-rbac-proxy"
Aug 05 17:35:43.108077 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.108055    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.110815 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.110794    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cojeqbt7sddc7\""
Aug 05 17:35:43.110925 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.110846    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Aug 05 17:35:43.110925 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.110909    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Aug 05 17:35:43.111261 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.111248    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Aug 05 17:35:43.111346 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.111331    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Aug 05 17:35:43.116584 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.116568    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Aug 05 17:35:43.116928 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.116914    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Aug 05 17:35:43.117013 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.116992    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Aug 05 17:35:43.117395 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.117382    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Aug 05 17:35:43.119357 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.119342    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Aug 05 17:35:43.120235 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.120221    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Aug 05 17:35:43.120319 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.120265    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Aug 05 17:35:43.121489 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.121473    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-b2nbt\""
Aug 05 17:35:43.122113 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.122098    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Aug 05 17:35:43.141244 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.141211    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.141447 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.141255    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.141447 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.141284    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.141447 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.141309    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.141447 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.141354    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.141447 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.141377    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-config\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.141447 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.141419    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.141775 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.141504    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.141775 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.141620    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.141775 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.141737    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.141932 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.141766    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.141932 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.141816    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-web-config\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.141932 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.141867    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-config-out\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.141932 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.141913    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.142376 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.142354    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.142478 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.142411    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5fhl\" (UniqueName: \"kubernetes.io/projected/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-kube-api-access-k5fhl\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.142541 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.142504    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.142624 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.142606    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.142690 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.142675    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.142743 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.142701    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.149937 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.149912    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="132dda5b-3438-4121-90d5-b6f71da27477" path="/var/lib/kubelet/pods/132dda5b-3438-4121-90d5-b6f71da27477/volumes"
Aug 05 17:35:43.150941 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.150923    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 05 17:35:43.152217 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.152201    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Aug 05 17:35:43.161386 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.161369    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Aug 05 17:35:43.243133 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243046    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-config-out\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243133 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243087    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243133 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243109    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243133 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243124    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5fhl\" (UniqueName: \"kubernetes.io/projected/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-kube-api-access-k5fhl\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243484 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243233    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243484 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243303    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243484 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243336    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243484 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243363    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243484 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243394    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243484 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243426    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243484 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243454    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243484 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243480    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243859 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243510    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243859 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243539    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-config\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243859 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243563    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243859 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243593    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243859 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243597    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243859 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243632    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243859 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243657    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243859 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243680    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.243859 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.243705    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-web-config\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.244424 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.244051    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.244978 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.244950    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.246366 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.246340    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-config-out\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.246466 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.246437    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.246697 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.246667    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-config\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.246977 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.246953    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.248292 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.247705    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-web-config\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.248292 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.247847    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.248292 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.247941    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.248491 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.248307    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.248980 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.248956    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.249485 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.249458    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.249799 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.249778    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.250563 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.250541    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.250671 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.250650    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.250739 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.250717    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.251292 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.251270    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.251728 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.251708    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.253463 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.253447    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5fhl\" (UniqueName: \"kubernetes.io/projected/d0389b49-02c0-4ec9-82fd-e47ceb75ceca-kube-api-access-k5fhl\") pod \"prometheus-k8s-0\" (UID: \"d0389b49-02c0-4ec9-82fd-e47ceb75ceca\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.417728 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.417696    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:43.579763 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:43.579733    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 05 17:35:43.582993 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:35:43.582965    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0389b49_02c0_4ec9_82fd_e47ceb75ceca.slice/crio-f4b22eda30c256662cd25b5625c1edfadac685d3ae82e024da17b98d79798d67 WatchSource:0}: Error finding container f4b22eda30c256662cd25b5625c1edfadac685d3ae82e024da17b98d79798d67: Status 404 returned error can't find the container with id f4b22eda30c256662cd25b5625c1edfadac685d3ae82e024da17b98d79798d67
Aug 05 17:35:44.030083 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:44.030056    2569 generic.go:356] "Generic (PLEG): container finished" podID="d0389b49-02c0-4ec9-82fd-e47ceb75ceca" containerID="fc9ab6bb64ce1ee9cf749d3a9a3e36efa0fa56dea923b191ee75f54d41c20029" exitCode=0
Aug 05 17:35:44.030430 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:44.030131    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0389b49-02c0-4ec9-82fd-e47ceb75ceca","Type":"ContainerDied","Data":"fc9ab6bb64ce1ee9cf749d3a9a3e36efa0fa56dea923b191ee75f54d41c20029"}
Aug 05 17:35:44.030430 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:44.030204    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0389b49-02c0-4ec9-82fd-e47ceb75ceca","Type":"ContainerStarted","Data":"f4b22eda30c256662cd25b5625c1edfadac685d3ae82e024da17b98d79798d67"}
Aug 05 17:35:45.036875 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:45.036843    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0389b49-02c0-4ec9-82fd-e47ceb75ceca","Type":"ContainerStarted","Data":"d0db3b0012f428f83cccbcf33e73ea8e6295fb5c9e212bc1cd051d941ab59df0"}
Aug 05 17:35:45.037319 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:45.036914    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0389b49-02c0-4ec9-82fd-e47ceb75ceca","Type":"ContainerStarted","Data":"d5543da120d058d9f25b3f35d92835934527fc99c8f829cbfe26ffc1f63d2eec"}
Aug 05 17:35:45.037319 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:45.036941    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0389b49-02c0-4ec9-82fd-e47ceb75ceca","Type":"ContainerStarted","Data":"6131def69db4c7ce713265c4f732b6eed723af7d57bf0833b1a5b290933f3614"}
Aug 05 17:35:45.037319 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:45.036970    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0389b49-02c0-4ec9-82fd-e47ceb75ceca","Type":"ContainerStarted","Data":"4201b28fbabed25c8e601ee5987b3d1bab41c19bf9aec4c7ff2f4330fb6c899a"}
Aug 05 17:35:45.037319 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:45.036993    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0389b49-02c0-4ec9-82fd-e47ceb75ceca","Type":"ContainerStarted","Data":"5597bed19d0b093268261411087d37bc0e9b17c4f7c5bfd3af8602844b778892"}
Aug 05 17:35:45.037319 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:45.037016    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0389b49-02c0-4ec9-82fd-e47ceb75ceca","Type":"ContainerStarted","Data":"98f1b3e3f2f5e923427ea64e48476e2b9a254c9d5c265985dff67d14d7c46651"}
Aug 05 17:35:45.074804 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:45.074754    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.055470559 podStartE2EDuration="2.074738845s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.268286ms" isStatefulPod=false podCreationTimestamp="2026-08-05 17:35:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:35:45.072294502 +0000 UTC m=+152.424969098" watchObservedRunningTime="2026-08-05 17:35:45.074738845 +0000 UTC m=+152.427413438"
Aug 05 17:35:47.242200 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.242150    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67bbc44bdf-hzlds"]
Aug 05 17:35:47.245981 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.245963    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.260603 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.260583    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67bbc44bdf-hzlds"]
Aug 05 17:35:47.276529 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.276500    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-console-config\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.276685 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.276572    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-oauth-serving-cert\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.276685 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.276601    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wbwj\" (UniqueName: \"kubernetes.io/projected/a63a3c02-c450-4fa6-abd7-3489eaded8dd-kube-api-access-4wbwj\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.276685 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.276632    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a63a3c02-c450-4fa6-abd7-3489eaded8dd-console-serving-cert\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.276685 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.276647    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-service-ca\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.276685 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.276685    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a63a3c02-c450-4fa6-abd7-3489eaded8dd-console-oauth-config\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.276879 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.276719    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-trusted-ca-bundle\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.377835 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.377798    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-oauth-serving-cert\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.377835 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.377833    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wbwj\" (UniqueName: \"kubernetes.io/projected/a63a3c02-c450-4fa6-abd7-3489eaded8dd-kube-api-access-4wbwj\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.378071 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.377859    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a63a3c02-c450-4fa6-abd7-3489eaded8dd-console-serving-cert\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.378071 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.377934    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-service-ca\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.378071 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.377969    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a63a3c02-c450-4fa6-abd7-3489eaded8dd-console-oauth-config\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.378071 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.377998    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-trusted-ca-bundle\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.378071 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.378047    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-console-config\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.378616 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.378592    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-oauth-serving-cert\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.378710 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.378666    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-service-ca\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.378710 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.378694    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-console-config\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.379021 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.379004    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-trusted-ca-bundle\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.380905 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.380878    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a63a3c02-c450-4fa6-abd7-3489eaded8dd-console-oauth-config\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.381112 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.381093    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a63a3c02-c450-4fa6-abd7-3489eaded8dd-console-serving-cert\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.387360 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.387327    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wbwj\" (UniqueName: \"kubernetes.io/projected/a63a3c02-c450-4fa6-abd7-3489eaded8dd-kube-api-access-4wbwj\") pod \"console-67bbc44bdf-hzlds\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") " pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.477348 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.477317    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:47.554936 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.554863    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:47.574935 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.574906    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:47.690206 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:47.690168    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67bbc44bdf-hzlds"]
Aug 05 17:35:47.692528 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:35:47.692502    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda63a3c02_c450_4fa6_abd7_3489eaded8dd.slice/crio-78ff87993b1ffc2b36f1067619e1dddc6e5c1abacc705c14af318caed0c3f56f WatchSource:0}: Error finding container 78ff87993b1ffc2b36f1067619e1dddc6e5c1abacc705c14af318caed0c3f56f: Status 404 returned error can't find the container with id 78ff87993b1ffc2b36f1067619e1dddc6e5c1abacc705c14af318caed0c3f56f
Aug 05 17:35:48.051622 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:48.051590    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67bbc44bdf-hzlds" event={"ID":"a63a3c02-c450-4fa6-abd7-3489eaded8dd","Type":"ContainerStarted","Data":"c70e8c059da14780cec71aa217d97696aa8b4e6a3dd816749d6c19e6d7a3f18c"}
Aug 05 17:35:48.051822 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:48.051643    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67bbc44bdf-hzlds" event={"ID":"a63a3c02-c450-4fa6-abd7-3489eaded8dd","Type":"ContainerStarted","Data":"78ff87993b1ffc2b36f1067619e1dddc6e5c1abacc705c14af318caed0c3f56f"}
Aug 05 17:35:48.059258 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:48.059233    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:48.075323 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:48.075259    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67bbc44bdf-hzlds" podStartSLOduration=1.075236587 podStartE2EDuration="1.075236587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:35:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:35:48.072610275 +0000 UTC m=+155.425284865" watchObservedRunningTime="2026-08-05 17:35:48.075236587 +0000 UTC m=+155.427911182"
Aug 05 17:35:48.160118 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:48.160075    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:48.418271 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:48.418240    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:35:57.554954 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:57.554924    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:57.555399 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:57.555237    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:57.559995 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:57.559973    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:58.099346 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:58.099314    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:35:58.152456 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:58.152428    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69c65ccfc5-z7nkj"]
Aug 05 17:35:58.282424 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:58.282396    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:58.383061 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:58.382977    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:58.482396 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:58.482358    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:58.552916 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:35:58.552891    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:36:23.177907 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.177840    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-69c65ccfc5-z7nkj" podUID="20caaf9f-d0d1-43f7-a283-941707f60e40" containerName="console" containerID="cri-o://87d647fb8afa3970204e2c4c40468f7b2227504373279a4e13c772c1730a4c0d" gracePeriod=15
Aug 05 17:36:23.420948 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.420925    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:36:23.479037 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.478956    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-trusted-ca-bundle\") pod \"20caaf9f-d0d1-43f7-a283-941707f60e40\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") "
Aug 05 17:36:23.479037 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.479005    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2z9z\" (UniqueName: \"kubernetes.io/projected/20caaf9f-d0d1-43f7-a283-941707f60e40-kube-api-access-r2z9z\") pod \"20caaf9f-d0d1-43f7-a283-941707f60e40\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") "
Aug 05 17:36:23.479037 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.479032    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-console-config\") pod \"20caaf9f-d0d1-43f7-a283-941707f60e40\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") "
Aug 05 17:36:23.479327 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.479077    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20caaf9f-d0d1-43f7-a283-941707f60e40-console-oauth-config\") pod \"20caaf9f-d0d1-43f7-a283-941707f60e40\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") "
Aug 05 17:36:23.479327 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.479120    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-oauth-serving-cert\") pod \"20caaf9f-d0d1-43f7-a283-941707f60e40\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") "
Aug 05 17:36:23.479327 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.479199    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20caaf9f-d0d1-43f7-a283-941707f60e40-console-serving-cert\") pod \"20caaf9f-d0d1-43f7-a283-941707f60e40\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") "
Aug 05 17:36:23.479327 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.479233    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-service-ca\") pod \"20caaf9f-d0d1-43f7-a283-941707f60e40\" (UID: \"20caaf9f-d0d1-43f7-a283-941707f60e40\") "
Aug 05 17:36:23.479536 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.479430    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "20caaf9f-d0d1-43f7-a283-941707f60e40" (UID: "20caaf9f-d0d1-43f7-a283-941707f60e40"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:36:23.479536 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.479504    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-trusted-ca-bundle\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:36:23.479536 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.479507    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-console-config" (OuterVolumeSpecName: "console-config") pod "20caaf9f-d0d1-43f7-a283-941707f60e40" (UID: "20caaf9f-d0d1-43f7-a283-941707f60e40"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:36:23.479678 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.479548    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "20caaf9f-d0d1-43f7-a283-941707f60e40" (UID: "20caaf9f-d0d1-43f7-a283-941707f60e40"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:36:23.479983 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.479904    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-service-ca" (OuterVolumeSpecName: "service-ca") pod "20caaf9f-d0d1-43f7-a283-941707f60e40" (UID: "20caaf9f-d0d1-43f7-a283-941707f60e40"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:36:23.481570 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.481539    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20caaf9f-d0d1-43f7-a283-941707f60e40-kube-api-access-r2z9z" (OuterVolumeSpecName: "kube-api-access-r2z9z") pod "20caaf9f-d0d1-43f7-a283-941707f60e40" (UID: "20caaf9f-d0d1-43f7-a283-941707f60e40"). InnerVolumeSpecName "kube-api-access-r2z9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:36:23.481570 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.481539    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20caaf9f-d0d1-43f7-a283-941707f60e40-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "20caaf9f-d0d1-43f7-a283-941707f60e40" (UID: "20caaf9f-d0d1-43f7-a283-941707f60e40"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:36:23.481711 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.481587    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20caaf9f-d0d1-43f7-a283-941707f60e40-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "20caaf9f-d0d1-43f7-a283-941707f60e40" (UID: "20caaf9f-d0d1-43f7-a283-941707f60e40"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:36:23.580933 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.580890    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20caaf9f-d0d1-43f7-a283-941707f60e40-console-oauth-config\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:36:23.580933 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.580925    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-oauth-serving-cert\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:36:23.580933 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.580935    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20caaf9f-d0d1-43f7-a283-941707f60e40-console-serving-cert\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:36:23.580933 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.580944    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-service-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:36:23.581205 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.580953    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2z9z\" (UniqueName: \"kubernetes.io/projected/20caaf9f-d0d1-43f7-a283-941707f60e40-kube-api-access-r2z9z\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:36:23.581205 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:23.580962    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20caaf9f-d0d1-43f7-a283-941707f60e40-console-config\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:36:24.199922 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:24.199885    2569 generic.go:356] "Generic (PLEG): container finished" podID="20caaf9f-d0d1-43f7-a283-941707f60e40" containerID="87d647fb8afa3970204e2c4c40468f7b2227504373279a4e13c772c1730a4c0d" exitCode=0
Aug 05 17:36:24.199922 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:24.199920    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69c65ccfc5-z7nkj" event={"ID":"20caaf9f-d0d1-43f7-a283-941707f60e40","Type":"ContainerDied","Data":"87d647fb8afa3970204e2c4c40468f7b2227504373279a4e13c772c1730a4c0d"}
Aug 05 17:36:24.200393 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:24.199936    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69c65ccfc5-z7nkj"
Aug 05 17:36:24.200393 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:24.199955    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69c65ccfc5-z7nkj" event={"ID":"20caaf9f-d0d1-43f7-a283-941707f60e40","Type":"ContainerDied","Data":"53ed9016c68ad3e4a67bb02b0db649b8b1d137b465b9be58c6aca18e62f75d7c"}
Aug 05 17:36:24.200393 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:24.199976    2569 scope.go:117] "RemoveContainer" containerID="87d647fb8afa3970204e2c4c40468f7b2227504373279a4e13c772c1730a4c0d"
Aug 05 17:36:24.208688 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:24.208671    2569 scope.go:117] "RemoveContainer" containerID="87d647fb8afa3970204e2c4c40468f7b2227504373279a4e13c772c1730a4c0d"
Aug 05 17:36:24.208930 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:36:24.208911    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87d647fb8afa3970204e2c4c40468f7b2227504373279a4e13c772c1730a4c0d\": container with ID starting with 87d647fb8afa3970204e2c4c40468f7b2227504373279a4e13c772c1730a4c0d not found: ID does not exist" containerID="87d647fb8afa3970204e2c4c40468f7b2227504373279a4e13c772c1730a4c0d"
Aug 05 17:36:24.209012 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:24.208936    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87d647fb8afa3970204e2c4c40468f7b2227504373279a4e13c772c1730a4c0d"} err="failed to get container status \"87d647fb8afa3970204e2c4c40468f7b2227504373279a4e13c772c1730a4c0d\": rpc error: code = NotFound desc = could not find container \"87d647fb8afa3970204e2c4c40468f7b2227504373279a4e13c772c1730a4c0d\": container with ID starting with 87d647fb8afa3970204e2c4c40468f7b2227504373279a4e13c772c1730a4c0d not found: ID does not exist"
Aug 05 17:36:24.229376 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:24.229355    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69c65ccfc5-z7nkj"]
Aug 05 17:36:24.242698 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:24.242677    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-69c65ccfc5-z7nkj"]
Aug 05 17:36:24.911650 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:24.911618    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:36:24.999418 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:24.999390    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:36:25.150049 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:25.150015    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20caaf9f-d0d1-43f7-a283-941707f60e40" path="/var/lib/kubelet/pods/20caaf9f-d0d1-43f7-a283-941707f60e40/volumes"
Aug 05 17:36:43.418252 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:43.418216    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:36:43.434064 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:43.434040    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:36:44.295702 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:44.295678    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:36:52.131123 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:52.131029    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:36:52.228730 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:52.228692    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:36:58.012875 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.012830    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-577d9b9fdf-dmcx2"]
Aug 05 17:36:58.013306 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.013172    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20caaf9f-d0d1-43f7-a283-941707f60e40" containerName="console"
Aug 05 17:36:58.013306 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.013202    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="20caaf9f-d0d1-43f7-a283-941707f60e40" containerName="console"
Aug 05 17:36:58.013306 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.013269    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="20caaf9f-d0d1-43f7-a283-941707f60e40" containerName="console"
Aug 05 17:36:58.016030 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.016012    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.037453 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.037426    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-577d9b9fdf-dmcx2"]
Aug 05 17:36:58.078715 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.078669    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:36:58.145418 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.145379    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:36:58.173586 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.173564    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-service-ca\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.173709 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.173611    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qknrl\" (UniqueName: \"kubernetes.io/projected/3859ed47-52a8-408b-8b10-c9771e05d349-kube-api-access-qknrl\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.173709 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.173635    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-oauth-serving-cert\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.173709 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.173652    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-trusted-ca-bundle\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.173709 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.173668    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-console-config\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.173709 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.173688    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3859ed47-52a8-408b-8b10-c9771e05d349-console-oauth-config\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.173870 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.173769    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3859ed47-52a8-408b-8b10-c9771e05d349-console-serving-cert\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.274585 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.274497    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qknrl\" (UniqueName: \"kubernetes.io/projected/3859ed47-52a8-408b-8b10-c9771e05d349-kube-api-access-qknrl\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.274585 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.274535    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-oauth-serving-cert\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.274585 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.274553    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-trusted-ca-bundle\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.274899 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.274875    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-console-config\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.274963 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.274942    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3859ed47-52a8-408b-8b10-c9771e05d349-console-oauth-config\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.275020 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.275003    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3859ed47-52a8-408b-8b10-c9771e05d349-console-serving-cert\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.275080 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.275065    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-service-ca\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.275261 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.275236    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-oauth-serving-cert\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.275407 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.275388    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-trusted-ca-bundle\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.275505 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.275468    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-console-config\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.275715 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.275698    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-service-ca\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.277551 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.277530    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3859ed47-52a8-408b-8b10-c9771e05d349-console-oauth-config\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.277683 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.277664    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3859ed47-52a8-408b-8b10-c9771e05d349-console-serving-cert\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.283785 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.283763    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qknrl\" (UniqueName: \"kubernetes.io/projected/3859ed47-52a8-408b-8b10-c9771e05d349-kube-api-access-qknrl\") pod \"console-577d9b9fdf-dmcx2\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") " pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.325764 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.325735    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:36:58.459635 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:58.459610    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-577d9b9fdf-dmcx2"]
Aug 05 17:36:58.462436 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:36:58.462404    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3859ed47_52a8_408b_8b10_c9771e05d349.slice/crio-db7c1c20baee8a620ee892eacc3a9da7a4526b21fd0737660d32626cfde37052 WatchSource:0}: Error finding container db7c1c20baee8a620ee892eacc3a9da7a4526b21fd0737660d32626cfde37052: Status 404 returned error can't find the container with id db7c1c20baee8a620ee892eacc3a9da7a4526b21fd0737660d32626cfde37052
Aug 05 17:36:59.343238 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:59.343197    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-577d9b9fdf-dmcx2" event={"ID":"3859ed47-52a8-408b-8b10-c9771e05d349","Type":"ContainerStarted","Data":"883358c92c6e6608a3a1dd215a2f87c8681e2d8c80cf5b3028b48f468ba03123"}
Aug 05 17:36:59.343708 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:59.343250    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-577d9b9fdf-dmcx2" event={"ID":"3859ed47-52a8-408b-8b10-c9771e05d349","Type":"ContainerStarted","Data":"db7c1c20baee8a620ee892eacc3a9da7a4526b21fd0737660d32626cfde37052"}
Aug 05 17:36:59.363952 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:36:59.363900    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-577d9b9fdf-dmcx2" podStartSLOduration=2.363884202 podStartE2EDuration="2.363884202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:36:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:36:59.3634295 +0000 UTC m=+226.716104095" watchObservedRunningTime="2026-08-05 17:36:59.363884202 +0000 UTC m=+226.716558796"
Aug 05 17:37:04.127915 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:04.127881    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:37:04.193216 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:04.193163    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:37:06.290034 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:06.290004    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:37:08.326477 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:08.326442    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:37:08.326881 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:08.326502    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:37:08.331308 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:08.331285    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:37:08.384210 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:08.384166    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:37:08.444718 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:08.444688    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67bbc44bdf-hzlds"]
Aug 05 17:37:08.590042 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:08.589972    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:37:08.691536 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:08.691507    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:37:08.791726 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:08.791692    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:37:15.099263 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:15.099229    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:37:15.198691 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:15.198658    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:37:15.264586 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:15.264563    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:37:33.464074 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.464011    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-67bbc44bdf-hzlds" podUID="a63a3c02-c450-4fa6-abd7-3489eaded8dd" containerName="console" containerID="cri-o://c70e8c059da14780cec71aa217d97696aa8b4e6a3dd816749d6c19e6d7a3f18c" gracePeriod=15
Aug 05 17:37:33.707054 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.707032    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:37:33.761950 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.761891    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-service-ca\") pod \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") "
Aug 05 17:37:33.761950 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.761933    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a63a3c02-c450-4fa6-abd7-3489eaded8dd-console-serving-cert\") pod \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") "
Aug 05 17:37:33.762097 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.761958    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-trusted-ca-bundle\") pod \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") "
Aug 05 17:37:33.762097 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.762077    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-console-config\") pod \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") "
Aug 05 17:37:33.762169 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.762112    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-oauth-serving-cert\") pod \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") "
Aug 05 17:37:33.762169 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.762144    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a63a3c02-c450-4fa6-abd7-3489eaded8dd-console-oauth-config\") pod \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") "
Aug 05 17:37:33.762308 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.762209    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wbwj\" (UniqueName: \"kubernetes.io/projected/a63a3c02-c450-4fa6-abd7-3489eaded8dd-kube-api-access-4wbwj\") pod \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\" (UID: \"a63a3c02-c450-4fa6-abd7-3489eaded8dd\") "
Aug 05 17:37:33.762486 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.762369    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a63a3c02-c450-4fa6-abd7-3489eaded8dd" (UID: "a63a3c02-c450-4fa6-abd7-3489eaded8dd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:37:33.762612 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.762447    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-service-ca" (OuterVolumeSpecName: "service-ca") pod "a63a3c02-c450-4fa6-abd7-3489eaded8dd" (UID: "a63a3c02-c450-4fa6-abd7-3489eaded8dd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:37:33.762612 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.762489    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-console-config" (OuterVolumeSpecName: "console-config") pod "a63a3c02-c450-4fa6-abd7-3489eaded8dd" (UID: "a63a3c02-c450-4fa6-abd7-3489eaded8dd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:37:33.762705 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.762550    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a63a3c02-c450-4fa6-abd7-3489eaded8dd" (UID: "a63a3c02-c450-4fa6-abd7-3489eaded8dd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:37:33.764283 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.764261    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a63a3c02-c450-4fa6-abd7-3489eaded8dd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a63a3c02-c450-4fa6-abd7-3489eaded8dd" (UID: "a63a3c02-c450-4fa6-abd7-3489eaded8dd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:37:33.764410 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.764389    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a63a3c02-c450-4fa6-abd7-3489eaded8dd-kube-api-access-4wbwj" (OuterVolumeSpecName: "kube-api-access-4wbwj") pod "a63a3c02-c450-4fa6-abd7-3489eaded8dd" (UID: "a63a3c02-c450-4fa6-abd7-3489eaded8dd"). InnerVolumeSpecName "kube-api-access-4wbwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:37:33.764448 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.764429    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a63a3c02-c450-4fa6-abd7-3489eaded8dd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a63a3c02-c450-4fa6-abd7-3489eaded8dd" (UID: "a63a3c02-c450-4fa6-abd7-3489eaded8dd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:37:33.863167 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.863119    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wbwj\" (UniqueName: \"kubernetes.io/projected/a63a3c02-c450-4fa6-abd7-3489eaded8dd-kube-api-access-4wbwj\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:37:33.863167 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.863140    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-service-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:37:33.863167 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.863149    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a63a3c02-c450-4fa6-abd7-3489eaded8dd-console-serving-cert\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:37:33.863167 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.863158    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-trusted-ca-bundle\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:37:33.863167 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.863167    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-console-config\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:37:33.863384 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.863195    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a63a3c02-c450-4fa6-abd7-3489eaded8dd-oauth-serving-cert\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:37:33.863384 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:33.863204    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a63a3c02-c450-4fa6-abd7-3489eaded8dd-console-oauth-config\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:37:34.484474 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:34.484440    2569 generic.go:356] "Generic (PLEG): container finished" podID="a63a3c02-c450-4fa6-abd7-3489eaded8dd" containerID="c70e8c059da14780cec71aa217d97696aa8b4e6a3dd816749d6c19e6d7a3f18c" exitCode=0
Aug 05 17:37:34.484899 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:34.484487    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67bbc44bdf-hzlds" event={"ID":"a63a3c02-c450-4fa6-abd7-3489eaded8dd","Type":"ContainerDied","Data":"c70e8c059da14780cec71aa217d97696aa8b4e6a3dd816749d6c19e6d7a3f18c"}
Aug 05 17:37:34.484899 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:34.484500    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67bbc44bdf-hzlds"
Aug 05 17:37:34.484899 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:34.484539    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67bbc44bdf-hzlds" event={"ID":"a63a3c02-c450-4fa6-abd7-3489eaded8dd","Type":"ContainerDied","Data":"78ff87993b1ffc2b36f1067619e1dddc6e5c1abacc705c14af318caed0c3f56f"}
Aug 05 17:37:34.484899 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:34.484569    2569 scope.go:117] "RemoveContainer" containerID="c70e8c059da14780cec71aa217d97696aa8b4e6a3dd816749d6c19e6d7a3f18c"
Aug 05 17:37:34.493904 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:34.493887    2569 scope.go:117] "RemoveContainer" containerID="c70e8c059da14780cec71aa217d97696aa8b4e6a3dd816749d6c19e6d7a3f18c"
Aug 05 17:37:34.494162 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:37:34.494143    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c70e8c059da14780cec71aa217d97696aa8b4e6a3dd816749d6c19e6d7a3f18c\": container with ID starting with c70e8c059da14780cec71aa217d97696aa8b4e6a3dd816749d6c19e6d7a3f18c not found: ID does not exist" containerID="c70e8c059da14780cec71aa217d97696aa8b4e6a3dd816749d6c19e6d7a3f18c"
Aug 05 17:37:34.494247 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:34.494168    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c70e8c059da14780cec71aa217d97696aa8b4e6a3dd816749d6c19e6d7a3f18c"} err="failed to get container status \"c70e8c059da14780cec71aa217d97696aa8b4e6a3dd816749d6c19e6d7a3f18c\": rpc error: code = NotFound desc = could not find container \"c70e8c059da14780cec71aa217d97696aa8b4e6a3dd816749d6c19e6d7a3f18c\": container with ID starting with c70e8c059da14780cec71aa217d97696aa8b4e6a3dd816749d6c19e6d7a3f18c not found: ID does not exist"
Aug 05 17:37:34.505661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:34.505632    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67bbc44bdf-hzlds"]
Aug 05 17:37:34.508738 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:34.508719    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67bbc44bdf-hzlds"]
Aug 05 17:37:35.150280 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:35.150248    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a63a3c02-c450-4fa6-abd7-3489eaded8dd" path="/var/lib/kubelet/pods/a63a3c02-c450-4fa6-abd7-3489eaded8dd/volumes"
Aug 05 17:37:50.735917 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:50.735877    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:37:50.836418 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:37:50.836393    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:13.009062 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:13.009021    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:13.014461 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:13.014430    2569 kubelet.go:1648] "Image garbage collection succeeded"
Aug 05 17:38:14.733817 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:14.733786    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:17.058619 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:17.058588    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:17.157065 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:17.157026    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:20.185683 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:20.185630    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:20.361881 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:20.361853    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:20.459397 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:20.459323    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:22.669542 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:22.669513    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:22.882597 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:22.882551    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:23.387909 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:23.387870    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:23.515454 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:23.515433    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:27.646320 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:27.646292    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-qnfd4"]
Aug 05 17:38:27.646663 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:27.646602    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a63a3c02-c450-4fa6-abd7-3489eaded8dd" containerName="console"
Aug 05 17:38:27.646663 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:27.646613    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a63a3c02-c450-4fa6-abd7-3489eaded8dd" containerName="console"
Aug 05 17:38:27.646730 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:27.646672    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a63a3c02-c450-4fa6-abd7-3489eaded8dd" containerName="console"
Aug 05 17:38:27.649376 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:27.649358    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qnfd4"
Aug 05 17:38:27.652042 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:27.652015    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Aug 05 17:38:27.652225 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:27.652208    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Aug 05 17:38:27.652389 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:27.652376    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-2k5nk\""
Aug 05 17:38:27.660074 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:27.660056    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-qnfd4"]
Aug 05 17:38:27.748491 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:27.748469    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/556a00d7-712b-4ef4-b027-6543e8e6a1bc-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-qnfd4\" (UID: \"556a00d7-712b-4ef4-b027-6543e8e6a1bc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qnfd4"
Aug 05 17:38:27.748577 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:27.748501    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nxkh\" (UniqueName: \"kubernetes.io/projected/556a00d7-712b-4ef4-b027-6543e8e6a1bc-kube-api-access-2nxkh\") pod \"cert-manager-cainjector-559cdd8c87-qnfd4\" (UID: \"556a00d7-712b-4ef4-b027-6543e8e6a1bc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qnfd4"
Aug 05 17:38:27.849545 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:27.849524    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/556a00d7-712b-4ef4-b027-6543e8e6a1bc-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-qnfd4\" (UID: \"556a00d7-712b-4ef4-b027-6543e8e6a1bc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qnfd4"
Aug 05 17:38:27.849627 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:27.849556    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nxkh\" (UniqueName: \"kubernetes.io/projected/556a00d7-712b-4ef4-b027-6543e8e6a1bc-kube-api-access-2nxkh\") pod \"cert-manager-cainjector-559cdd8c87-qnfd4\" (UID: \"556a00d7-712b-4ef4-b027-6543e8e6a1bc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qnfd4"
Aug 05 17:38:27.858589 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:27.858562    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/556a00d7-712b-4ef4-b027-6543e8e6a1bc-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-qnfd4\" (UID: \"556a00d7-712b-4ef4-b027-6543e8e6a1bc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qnfd4"
Aug 05 17:38:27.858987 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:27.858971    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nxkh\" (UniqueName: \"kubernetes.io/projected/556a00d7-712b-4ef4-b027-6543e8e6a1bc-kube-api-access-2nxkh\") pod \"cert-manager-cainjector-559cdd8c87-qnfd4\" (UID: \"556a00d7-712b-4ef4-b027-6543e8e6a1bc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qnfd4"
Aug 05 17:38:27.967774 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:27.967728    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qnfd4"
Aug 05 17:38:28.091646 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:28.091623    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-qnfd4"]
Aug 05 17:38:28.094170 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:38:28.094140    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod556a00d7_712b_4ef4_b027_6543e8e6a1bc.slice/crio-1faf4ca1a12dac5fbde09c92de12b07385d38fc00e2776f3273f54aa195ce95a WatchSource:0}: Error finding container 1faf4ca1a12dac5fbde09c92de12b07385d38fc00e2776f3273f54aa195ce95a: Status 404 returned error can't find the container with id 1faf4ca1a12dac5fbde09c92de12b07385d38fc00e2776f3273f54aa195ce95a
Aug 05 17:38:28.095992 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:28.095977    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:38:28.699737 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:28.699704    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qnfd4" event={"ID":"556a00d7-712b-4ef4-b027-6543e8e6a1bc","Type":"ContainerStarted","Data":"1faf4ca1a12dac5fbde09c92de12b07385d38fc00e2776f3273f54aa195ce95a"}
Aug 05 17:38:29.884325 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:29.884279    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:30.943465 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:30.943433    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-62cq9"]
Aug 05 17:38:30.947004 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:30.946985    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-62cq9"
Aug 05 17:38:30.950112 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:30.950066    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-khxc2\""
Aug 05 17:38:30.957466 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:30.957414    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-62cq9"]
Aug 05 17:38:31.080235 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:31.080211    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7776\" (UniqueName: \"kubernetes.io/projected/2f364db1-4bc2-4271-8486-ce72ca70d8a7-kube-api-access-b7776\") pod \"cert-manager-55b4d954d5-62cq9\" (UID: \"2f364db1-4bc2-4271-8486-ce72ca70d8a7\") " pod="cert-manager/cert-manager-55b4d954d5-62cq9"
Aug 05 17:38:31.080328 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:31.080268    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f364db1-4bc2-4271-8486-ce72ca70d8a7-bound-sa-token\") pod \"cert-manager-55b4d954d5-62cq9\" (UID: \"2f364db1-4bc2-4271-8486-ce72ca70d8a7\") " pod="cert-manager/cert-manager-55b4d954d5-62cq9"
Aug 05 17:38:31.181223 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:31.181158    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7776\" (UniqueName: \"kubernetes.io/projected/2f364db1-4bc2-4271-8486-ce72ca70d8a7-kube-api-access-b7776\") pod \"cert-manager-55b4d954d5-62cq9\" (UID: \"2f364db1-4bc2-4271-8486-ce72ca70d8a7\") " pod="cert-manager/cert-manager-55b4d954d5-62cq9"
Aug 05 17:38:31.181429 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:31.181255    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f364db1-4bc2-4271-8486-ce72ca70d8a7-bound-sa-token\") pod \"cert-manager-55b4d954d5-62cq9\" (UID: \"2f364db1-4bc2-4271-8486-ce72ca70d8a7\") " pod="cert-manager/cert-manager-55b4d954d5-62cq9"
Aug 05 17:38:31.196513 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:31.196455    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f364db1-4bc2-4271-8486-ce72ca70d8a7-bound-sa-token\") pod \"cert-manager-55b4d954d5-62cq9\" (UID: \"2f364db1-4bc2-4271-8486-ce72ca70d8a7\") " pod="cert-manager/cert-manager-55b4d954d5-62cq9"
Aug 05 17:38:31.196609 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:31.196514    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7776\" (UniqueName: \"kubernetes.io/projected/2f364db1-4bc2-4271-8486-ce72ca70d8a7-kube-api-access-b7776\") pod \"cert-manager-55b4d954d5-62cq9\" (UID: \"2f364db1-4bc2-4271-8486-ce72ca70d8a7\") " pod="cert-manager/cert-manager-55b4d954d5-62cq9"
Aug 05 17:38:31.258889 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:31.258860    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-62cq9"
Aug 05 17:38:31.403643 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:31.403611    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-62cq9"]
Aug 05 17:38:31.407122 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:38:31.407091    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f364db1_4bc2_4271_8486_ce72ca70d8a7.slice/crio-197103a4d038e86b287d9b1811d6410e4f04f0d669de2b194efc373331c92a6d WatchSource:0}: Error finding container 197103a4d038e86b287d9b1811d6410e4f04f0d669de2b194efc373331c92a6d: Status 404 returned error can't find the container with id 197103a4d038e86b287d9b1811d6410e4f04f0d669de2b194efc373331c92a6d
Aug 05 17:38:31.715720 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:31.715627    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-62cq9" event={"ID":"2f364db1-4bc2-4271-8486-ce72ca70d8a7","Type":"ContainerStarted","Data":"ebaddc59a11a05d620ad8c3c1680e73365cd334aad1afd78876e079f657e087d"}
Aug 05 17:38:31.715720 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:31.715677    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-62cq9" event={"ID":"2f364db1-4bc2-4271-8486-ce72ca70d8a7","Type":"ContainerStarted","Data":"197103a4d038e86b287d9b1811d6410e4f04f0d669de2b194efc373331c92a6d"}
Aug 05 17:38:31.717059 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:31.717035    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qnfd4" event={"ID":"556a00d7-712b-4ef4-b027-6543e8e6a1bc","Type":"ContainerStarted","Data":"7a14ce496f5b6926db76739f37c737c1000f6b2bb62f98b616aeada80722f78b"}
Aug 05 17:38:31.758686 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:31.758642    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-62cq9" podStartSLOduration=1.7586284220000001 podStartE2EDuration="1.758628422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:38:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:38:31.758097415 +0000 UTC m=+319.110772006" watchObservedRunningTime="2026-08-05 17:38:31.758628422 +0000 UTC m=+319.111303015"
Aug 05 17:38:31.814344 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:31.814298    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qnfd4" podStartSLOduration=1.8408017060000001 podStartE2EDuration="4.814283636s" totalImagesPullingTime="2.97348193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:38:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:38:31.812725205 +0000 UTC m=+319.165399797" watchObservedRunningTime="2026-08-05 17:38:31.814283636 +0000 UTC m=+319.166958228"
Aug 05 17:38:31.991143 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:31.991068    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:32.113917 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:32.113889    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:36.347076 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:36.347045    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:36.438453 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:36.438428    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:41.547216 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:41.547164    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:41.647713 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:41.647688    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:43.274043 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:43.274015    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:59.773659 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.773620    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:59.805619 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.805591    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd"]
Aug 05 17:38:59.807760 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.807747    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd"
Aug 05 17:38:59.810577 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.810560    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Aug 05 17:38:59.810715 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.810682    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Aug 05 17:38:59.811738 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.811720    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Aug 05 17:38:59.811821 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.811720    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Aug 05 17:38:59.811865 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.811834    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hqhjq\""
Aug 05 17:38:59.812326 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.812311    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Aug 05 17:38:59.820450 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.820433    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd"]
Aug 05 17:38:59.875000 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.874978    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:59.884044 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.884024    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/40522c5b-a3f8-4343-9755-0795762c9808-manager-config\") pod \"lws-controller-manager-dd9d7cdcd-mwfrd\" (UID: \"40522c5b-a3f8-4343-9755-0795762c9808\") " pod="openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd"
Aug 05 17:38:59.884123 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.884053    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvfmb\" (UniqueName: \"kubernetes.io/projected/40522c5b-a3f8-4343-9755-0795762c9808-kube-api-access-gvfmb\") pod \"lws-controller-manager-dd9d7cdcd-mwfrd\" (UID: \"40522c5b-a3f8-4343-9755-0795762c9808\") " pod="openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd"
Aug 05 17:38:59.884123 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.884097    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/40522c5b-a3f8-4343-9755-0795762c9808-metrics-cert\") pod \"lws-controller-manager-dd9d7cdcd-mwfrd\" (UID: \"40522c5b-a3f8-4343-9755-0795762c9808\") " pod="openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd"
Aug 05 17:38:59.884242 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.884197    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/40522c5b-a3f8-4343-9755-0795762c9808-cert\") pod \"lws-controller-manager-dd9d7cdcd-mwfrd\" (UID: \"40522c5b-a3f8-4343-9755-0795762c9808\") " pod="openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd"
Aug 05 17:38:59.984534 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.984508    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/40522c5b-a3f8-4343-9755-0795762c9808-cert\") pod \"lws-controller-manager-dd9d7cdcd-mwfrd\" (UID: \"40522c5b-a3f8-4343-9755-0795762c9808\") " pod="openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd"
Aug 05 17:38:59.984625 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.984544    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/40522c5b-a3f8-4343-9755-0795762c9808-manager-config\") pod \"lws-controller-manager-dd9d7cdcd-mwfrd\" (UID: \"40522c5b-a3f8-4343-9755-0795762c9808\") " pod="openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd"
Aug 05 17:38:59.984625 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.984566    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvfmb\" (UniqueName: \"kubernetes.io/projected/40522c5b-a3f8-4343-9755-0795762c9808-kube-api-access-gvfmb\") pod \"lws-controller-manager-dd9d7cdcd-mwfrd\" (UID: \"40522c5b-a3f8-4343-9755-0795762c9808\") " pod="openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd"
Aug 05 17:38:59.984625 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.984615    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/40522c5b-a3f8-4343-9755-0795762c9808-metrics-cert\") pod \"lws-controller-manager-dd9d7cdcd-mwfrd\" (UID: \"40522c5b-a3f8-4343-9755-0795762c9808\") " pod="openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd"
Aug 05 17:38:59.985118 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.985095    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/40522c5b-a3f8-4343-9755-0795762c9808-manager-config\") pod \"lws-controller-manager-dd9d7cdcd-mwfrd\" (UID: \"40522c5b-a3f8-4343-9755-0795762c9808\") " pod="openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd"
Aug 05 17:38:59.987087 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.987063    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/40522c5b-a3f8-4343-9755-0795762c9808-cert\") pod \"lws-controller-manager-dd9d7cdcd-mwfrd\" (UID: \"40522c5b-a3f8-4343-9755-0795762c9808\") " pod="openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd"
Aug 05 17:38:59.987158 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.987138    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/40522c5b-a3f8-4343-9755-0795762c9808-metrics-cert\") pod \"lws-controller-manager-dd9d7cdcd-mwfrd\" (UID: \"40522c5b-a3f8-4343-9755-0795762c9808\") " pod="openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd"
Aug 05 17:38:59.997679 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:38:59.997658    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvfmb\" (UniqueName: \"kubernetes.io/projected/40522c5b-a3f8-4343-9755-0795762c9808-kube-api-access-gvfmb\") pod \"lws-controller-manager-dd9d7cdcd-mwfrd\" (UID: \"40522c5b-a3f8-4343-9755-0795762c9808\") " pod="openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd"
Aug 05 17:39:00.117290 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:00.117255    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd"
Aug 05 17:39:00.252477 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:00.252304    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd"]
Aug 05 17:39:00.255282 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:39:00.255248    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40522c5b_a3f8_4343_9755_0795762c9808.slice/crio-f02dcc5cf08edd90997668270f68aaf9b4db2a1209236ce2c12b3d913d5cedaa WatchSource:0}: Error finding container f02dcc5cf08edd90997668270f68aaf9b4db2a1209236ce2c12b3d913d5cedaa: Status 404 returned error can't find the container with id f02dcc5cf08edd90997668270f68aaf9b4db2a1209236ce2c12b3d913d5cedaa
Aug 05 17:39:00.267019 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:00.266999    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:00.363975 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:00.363950    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:00.830490 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:00.830464    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd" event={"ID":"40522c5b-a3f8-4343-9755-0795762c9808","Type":"ContainerStarted","Data":"f02dcc5cf08edd90997668270f68aaf9b4db2a1209236ce2c12b3d913d5cedaa"}
Aug 05 17:39:00.936331 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:00.936300    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:01.178730 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.178662    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:01.279312 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.279287    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:01.366834 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.366811    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"]
Aug 05 17:39:01.369278 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.369265    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"
Aug 05 17:39:01.371803 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.371786    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Aug 05 17:39:01.372472 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.372447    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Aug 05 17:39:01.372578 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.372492    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Aug 05 17:39:01.372578 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.372501    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Aug 05 17:39:01.372790 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.372776    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xmkgp\""
Aug 05 17:39:01.390656 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.390635    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"]
Aug 05 17:39:01.395559 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.395538    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/db10024a-4454-49c0-aa8a-c388c8290bf4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-856dccdfcf-lbzpc\" (UID: \"db10024a-4454-49c0-aa8a-c388c8290bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"
Aug 05 17:39:01.395648 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.395604    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/db10024a-4454-49c0-aa8a-c388c8290bf4-manifests\") pod \"opendatahub-operator-controller-manager-856dccdfcf-lbzpc\" (UID: \"db10024a-4454-49c0-aa8a-c388c8290bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"
Aug 05 17:39:01.395648 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.395630    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db10024a-4454-49c0-aa8a-c388c8290bf4-webhook-cert\") pod \"opendatahub-operator-controller-manager-856dccdfcf-lbzpc\" (UID: \"db10024a-4454-49c0-aa8a-c388c8290bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"
Aug 05 17:39:01.395829 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.395657    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kp65\" (UniqueName: \"kubernetes.io/projected/db10024a-4454-49c0-aa8a-c388c8290bf4-kube-api-access-2kp65\") pod \"opendatahub-operator-controller-manager-856dccdfcf-lbzpc\" (UID: \"db10024a-4454-49c0-aa8a-c388c8290bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"
Aug 05 17:39:01.395829 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.395727    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db10024a-4454-49c0-aa8a-c388c8290bf4-tmp\") pod \"opendatahub-operator-controller-manager-856dccdfcf-lbzpc\" (UID: \"db10024a-4454-49c0-aa8a-c388c8290bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"
Aug 05 17:39:01.496891 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.496808    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/db10024a-4454-49c0-aa8a-c388c8290bf4-manifests\") pod \"opendatahub-operator-controller-manager-856dccdfcf-lbzpc\" (UID: \"db10024a-4454-49c0-aa8a-c388c8290bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"
Aug 05 17:39:01.496891 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.496844    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db10024a-4454-49c0-aa8a-c388c8290bf4-webhook-cert\") pod \"opendatahub-operator-controller-manager-856dccdfcf-lbzpc\" (UID: \"db10024a-4454-49c0-aa8a-c388c8290bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"
Aug 05 17:39:01.497102 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.496894    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kp65\" (UniqueName: \"kubernetes.io/projected/db10024a-4454-49c0-aa8a-c388c8290bf4-kube-api-access-2kp65\") pod \"opendatahub-operator-controller-manager-856dccdfcf-lbzpc\" (UID: \"db10024a-4454-49c0-aa8a-c388c8290bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"
Aug 05 17:39:01.497102 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.496937    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db10024a-4454-49c0-aa8a-c388c8290bf4-tmp\") pod \"opendatahub-operator-controller-manager-856dccdfcf-lbzpc\" (UID: \"db10024a-4454-49c0-aa8a-c388c8290bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"
Aug 05 17:39:01.497102 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.496976    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/db10024a-4454-49c0-aa8a-c388c8290bf4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-856dccdfcf-lbzpc\" (UID: \"db10024a-4454-49c0-aa8a-c388c8290bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"
Aug 05 17:39:01.497306 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.497278    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/db10024a-4454-49c0-aa8a-c388c8290bf4-manifests\") pod \"opendatahub-operator-controller-manager-856dccdfcf-lbzpc\" (UID: \"db10024a-4454-49c0-aa8a-c388c8290bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"
Aug 05 17:39:01.499222 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.499199    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db10024a-4454-49c0-aa8a-c388c8290bf4-tmp\") pod \"opendatahub-operator-controller-manager-856dccdfcf-lbzpc\" (UID: \"db10024a-4454-49c0-aa8a-c388c8290bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"
Aug 05 17:39:01.499413 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.499398    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/db10024a-4454-49c0-aa8a-c388c8290bf4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-856dccdfcf-lbzpc\" (UID: \"db10024a-4454-49c0-aa8a-c388c8290bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"
Aug 05 17:39:01.499456 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.499443    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db10024a-4454-49c0-aa8a-c388c8290bf4-webhook-cert\") pod \"opendatahub-operator-controller-manager-856dccdfcf-lbzpc\" (UID: \"db10024a-4454-49c0-aa8a-c388c8290bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"
Aug 05 17:39:01.512618 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.512587    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kp65\" (UniqueName: \"kubernetes.io/projected/db10024a-4454-49c0-aa8a-c388c8290bf4-kube-api-access-2kp65\") pod \"opendatahub-operator-controller-manager-856dccdfcf-lbzpc\" (UID: \"db10024a-4454-49c0-aa8a-c388c8290bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"
Aug 05 17:39:01.519812 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.519789    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:01.616247 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.616223    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:01.681430 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.681410    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"
Aug 05 17:39:01.719531 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.719501    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:01.820288 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.820263    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:01.820717 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.820698    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"]
Aug 05 17:39:01.822919 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:39:01.822896    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb10024a_4454_49c0_aa8a_c388c8290bf4.slice/crio-15f1ee6881c436fcc75b94073ae4ed4a7a7774d107f01310c4a26d00d91f86db WatchSource:0}: Error finding container 15f1ee6881c436fcc75b94073ae4ed4a7a7774d107f01310c4a26d00d91f86db: Status 404 returned error can't find the container with id 15f1ee6881c436fcc75b94073ae4ed4a7a7774d107f01310c4a26d00d91f86db
Aug 05 17:39:01.836560 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.836538    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc" event={"ID":"db10024a-4454-49c0-aa8a-c388c8290bf4","Type":"ContainerStarted","Data":"15f1ee6881c436fcc75b94073ae4ed4a7a7774d107f01310c4a26d00d91f86db"}
Aug 05 17:39:01.916917 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:01.916841    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:02.026319 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:02.026247    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:06.862827 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:06.862788    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc" event={"ID":"db10024a-4454-49c0-aa8a-c388c8290bf4","Type":"ContainerStarted","Data":"50a8d3ba5593695ff41041187cd2be11a541b4e2531aa92e8b1b54c136fa7d2c"}
Aug 05 17:39:06.864310 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:06.864287    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd" event={"ID":"40522c5b-a3f8-4343-9755-0795762c9808","Type":"ContainerStarted","Data":"83fa82d084f8935d165f96d4d04caffa905a7e2af35d4968f16f599869602ca6"}
Aug 05 17:39:06.864423 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:06.864338    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd"
Aug 05 17:39:06.913028 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:06.912982    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd" podStartSLOduration=2.17357595 podStartE2EDuration="7.912968779s" totalImagesPullingTime="5.739392829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:38:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:39:06.912443251 +0000 UTC m=+354.265117846" watchObservedRunningTime="2026-08-05 17:39:06.912968779 +0000 UTC m=+354.265643373"
Aug 05 17:39:07.873947 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:07.873910    2569 generic.go:356] "Generic (PLEG): container finished" podID="db10024a-4454-49c0-aa8a-c388c8290bf4" containerID="50a8d3ba5593695ff41041187cd2be11a541b4e2531aa92e8b1b54c136fa7d2c" exitCode=0
Aug 05 17:39:07.874353 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:07.873974    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc" event={"ID":"db10024a-4454-49c0-aa8a-c388c8290bf4","Type":"ContainerDied","Data":"50a8d3ba5593695ff41041187cd2be11a541b4e2531aa92e8b1b54c136fa7d2c"}
Aug 05 17:39:08.880866 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:08.880821    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc" event={"ID":"db10024a-4454-49c0-aa8a-c388c8290bf4","Type":"ContainerStarted","Data":"279824a06193bb0fe63ed94a8d6fcb95b4f32a199d1b7e6d0d559319a3096cae"}
Aug 05 17:39:08.881264 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:08.880942    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"
Aug 05 17:39:08.917488 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:08.917445    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc" podStartSLOduration=1.7843513039999999 podStartE2EDuration="7.917431225s" totalImagesPullingTime="5.051418643s" totalInitContainerRuntime="1.081661278s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:39:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:39:08.914877288 +0000 UTC m=+356.267551884" watchObservedRunningTime="2026-08-05 17:39:08.917431225 +0000 UTC m=+356.270105818"
Aug 05 17:39:13.141486 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:13.141439    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:13.227056 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:13.227024    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:17.876745 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:17.876709    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-dd9d7cdcd-mwfrd"
Aug 05 17:39:19.887470 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:19.887438    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-856dccdfcf-lbzpc"
Aug 05 17:39:21.728282 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:21.728244    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:21.837795 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:21.837764    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:22.585023 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:22.584993    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:22.790156 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:22.790126    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-57558c7678-8bjnz"]
Aug 05 17:39:22.796195 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:22.796159    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57558c7678-8bjnz"
Aug 05 17:39:22.809606 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:22.809584    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Aug 05 17:39:22.809725 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:22.809581    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Aug 05 17:39:22.809725 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:22.809584    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Aug 05 17:39:22.811842 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:22.811821    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Aug 05 17:39:22.811936 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:22.811880    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-6g5hz\""
Aug 05 17:39:22.830136 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:22.830111    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57558c7678-8bjnz"]
Aug 05 17:39:22.886454 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:22.886374    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170-tls-certs\") pod \"kube-auth-proxy-57558c7678-8bjnz\" (UID: \"fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170\") " pod="openshift-ingress/kube-auth-proxy-57558c7678-8bjnz"
Aug 05 17:39:22.886454 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:22.886439    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170-tmp\") pod \"kube-auth-proxy-57558c7678-8bjnz\" (UID: \"fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170\") " pod="openshift-ingress/kube-auth-proxy-57558c7678-8bjnz"
Aug 05 17:39:22.886624 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:22.886457    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4stq\" (UniqueName: \"kubernetes.io/projected/fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170-kube-api-access-w4stq\") pod \"kube-auth-proxy-57558c7678-8bjnz\" (UID: \"fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170\") " pod="openshift-ingress/kube-auth-proxy-57558c7678-8bjnz"
Aug 05 17:39:22.938243 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:22.938205    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:22.987632 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:22.987596    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170-tls-certs\") pod \"kube-auth-proxy-57558c7678-8bjnz\" (UID: \"fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170\") " pod="openshift-ingress/kube-auth-proxy-57558c7678-8bjnz"
Aug 05 17:39:22.987825 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:22.987692    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170-tmp\") pod \"kube-auth-proxy-57558c7678-8bjnz\" (UID: \"fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170\") " pod="openshift-ingress/kube-auth-proxy-57558c7678-8bjnz"
Aug 05 17:39:22.987825 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:22.987721    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4stq\" (UniqueName: \"kubernetes.io/projected/fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170-kube-api-access-w4stq\") pod \"kube-auth-proxy-57558c7678-8bjnz\" (UID: \"fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170\") " pod="openshift-ingress/kube-auth-proxy-57558c7678-8bjnz"
Aug 05 17:39:22.990399 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:22.990341    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170-tmp\") pod \"kube-auth-proxy-57558c7678-8bjnz\" (UID: \"fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170\") " pod="openshift-ingress/kube-auth-proxy-57558c7678-8bjnz"
Aug 05 17:39:22.990700 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:22.990676    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170-tls-certs\") pod \"kube-auth-proxy-57558c7678-8bjnz\" (UID: \"fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170\") " pod="openshift-ingress/kube-auth-proxy-57558c7678-8bjnz"
Aug 05 17:39:23.028429 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:23.028399    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4stq\" (UniqueName: \"kubernetes.io/projected/fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170-kube-api-access-w4stq\") pod \"kube-auth-proxy-57558c7678-8bjnz\" (UID: \"fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170\") " pod="openshift-ingress/kube-auth-proxy-57558c7678-8bjnz"
Aug 05 17:39:23.093027 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:23.092967    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:23.105534 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:23.105512    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57558c7678-8bjnz"
Aug 05 17:39:23.182412 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:23.182345    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:23.265328 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:23.265237    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57558c7678-8bjnz"]
Aug 05 17:39:23.267588 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:39:23.267552    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa67e0f0_fbbf_4d26_8d7e_1d8b98b9d170.slice/crio-a73d7c4dcf552e951e6a4033d4849731bc19bf5949c8c74b2d0f2071a9cb6495 WatchSource:0}: Error finding container a73d7c4dcf552e951e6a4033d4849731bc19bf5949c8c74b2d0f2071a9cb6495: Status 404 returned error can't find the container with id a73d7c4dcf552e951e6a4033d4849731bc19bf5949c8c74b2d0f2071a9cb6495
Aug 05 17:39:23.281851 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:23.281826    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:23.505717 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:23.505644    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:23.875510 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:23.875482    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:23.948146 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:23.948107    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57558c7678-8bjnz" event={"ID":"fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170","Type":"ContainerStarted","Data":"a73d7c4dcf552e951e6a4033d4849731bc19bf5949c8c74b2d0f2071a9cb6495"}
Aug 05 17:39:23.961034 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:23.961002    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:24.065399 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:24.065369    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:24.215830 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:24.215644    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:24.433927 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:24.433861    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:24.517768 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:24.517691    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:25.686360 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:25.686328    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:26.965683 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:26.965650    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57558c7678-8bjnz" event={"ID":"fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170","Type":"ContainerStarted","Data":"f9a2db5595f07f0b690d51da66cb0d0b629dd1fdd63e63355d4cb69acb8c47cd"}
Aug 05 17:39:26.994780 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:26.994174    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-57558c7678-8bjnz" podStartSLOduration=2.151799878 podStartE2EDuration="4.994155022s" totalImagesPullingTime="2.842355144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:39:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:39:26.993968575 +0000 UTC m=+374.346643164" watchObservedRunningTime="2026-08-05 17:39:26.994155022 +0000 UTC m=+374.346829616"
Aug 05 17:39:31.314168 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:31.314125    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:31.434169 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:31.434137    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:32.531534 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:32.531496    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:32.870819 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:32.870788    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:33.014554 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:33.014519    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:35.677012 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:35.676976    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:35.770538 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:35.770508    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:35.869286 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:35.869258    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:35.968370 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:35.968310    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:36.896327 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:36.896295    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:37.030835 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:37.030803    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:37.128329 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:37.128300    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:37.233938 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:39:37.233869    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:23.363416 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:23.363375    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:23.451728 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:23.451693    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:30.727937 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:30.727907    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:32.697013 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:32.696981    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:32.797429 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:32.797394    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:40.639247 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:40.639150    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:41.438220 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:41.438171    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:44.662479 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:44.662445    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:44.991622 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:44.991530    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:45.102872 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:45.102836    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:46.657048 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:46.657017    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:46.912216 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:46.912100    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:47.077327 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:47.077295    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:47.663090 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:47.663053    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:48.053683 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:48.053599    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:48.142892 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:48.142858    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:51.050621 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:51.050583    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:52.127848 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:52.127819    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:52.585547 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:52.585522    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-2p7dn"]
Aug 05 17:40:52.588778 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:52.588763    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2p7dn"
Aug 05 17:40:52.593764 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:52.593743    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Aug 05 17:40:52.593878 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:52.593743    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Aug 05 17:40:52.594162 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:52.594141    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-wj8qd\""
Aug 05 17:40:52.594851 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:52.594834    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Aug 05 17:40:52.604318 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:52.604298    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-2p7dn"]
Aug 05 17:40:52.675562 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:52.675535    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw69n\" (UniqueName: \"kubernetes.io/projected/f328e280-a2cb-48e0-8c9f-10e2bc7af65b-kube-api-access-nw69n\") pod \"dns-operator-controller-manager-5985564776-2p7dn\" (UID: \"f328e280-a2cb-48e0-8c9f-10e2bc7af65b\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-2p7dn"
Aug 05 17:40:52.776699 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:52.776669    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw69n\" (UniqueName: \"kubernetes.io/projected/f328e280-a2cb-48e0-8c9f-10e2bc7af65b-kube-api-access-nw69n\") pod \"dns-operator-controller-manager-5985564776-2p7dn\" (UID: \"f328e280-a2cb-48e0-8c9f-10e2bc7af65b\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-2p7dn"
Aug 05 17:40:52.792409 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:52.792381    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw69n\" (UniqueName: \"kubernetes.io/projected/f328e280-a2cb-48e0-8c9f-10e2bc7af65b-kube-api-access-nw69n\") pod \"dns-operator-controller-manager-5985564776-2p7dn\" (UID: \"f328e280-a2cb-48e0-8c9f-10e2bc7af65b\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-2p7dn"
Aug 05 17:40:52.899579 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:52.899483    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2p7dn"
Aug 05 17:40:53.050137 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:53.050112    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-2p7dn"]
Aug 05 17:40:53.052553 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:40:53.052508    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf328e280_a2cb_48e0_8c9f_10e2bc7af65b.slice/crio-43a0d498ddeca72a5c5d0e9f98d460d5b9709a62d3ede5bec34c837075dc8d6e WatchSource:0}: Error finding container 43a0d498ddeca72a5c5d0e9f98d460d5b9709a62d3ede5bec34c837075dc8d6e: Status 404 returned error can't find the container with id 43a0d498ddeca72a5c5d0e9f98d460d5b9709a62d3ede5bec34c837075dc8d6e
Aug 05 17:40:53.348034 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:53.348006    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2p7dn" event={"ID":"f328e280-a2cb-48e0-8c9f-10e2bc7af65b","Type":"ContainerStarted","Data":"43a0d498ddeca72a5c5d0e9f98d460d5b9709a62d3ede5bec34c837075dc8d6e"}
Aug 05 17:40:54.056356 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:54.056327    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:54.115595 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:54.115553    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:54.454043 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:54.454014    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:54.684401 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:54.684371    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:57.375857 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:57.375821    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2p7dn" event={"ID":"f328e280-a2cb-48e0-8c9f-10e2bc7af65b","Type":"ContainerStarted","Data":"9c2a0204e20520e92e99212ab073471b0b0641afab550149fe57a1fb821d25fd"}
Aug 05 17:40:57.376347 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:57.375963    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2p7dn"
Aug 05 17:40:58.059008 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:58.058977    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:58.161767 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:40:58.161736    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:01.153427 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:01.153393    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:01.320308 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:01.320279    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:02.420579 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:02.420548    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:02.520314 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:02.520280    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:02.596850 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:02.596824    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:08.384517 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:08.384472    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2p7dn"
Aug 05 17:41:08.420427 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:08.420380    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2p7dn" podStartSLOduration=13.008697009 podStartE2EDuration="16.420367345s" totalImagesPullingTime="3.411670336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:40:57.418716101 +0000 UTC m=+464.771390694" watchObservedRunningTime="2026-08-05 17:41:08.420367345 +0000 UTC m=+475.773041939"
Aug 05 17:41:11.055120 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.055086    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"]
Aug 05 17:41:11.058379 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.058363    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"
Aug 05 17:41:11.061103 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.061079    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hfkjp\""
Aug 05 17:41:11.070811 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.070792    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"]
Aug 05 17:41:11.126979 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.126941    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23c743b9-c19c-446a-8fbd-68e4cd3c1a53-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-w926d\" (UID: \"23c743b9-c19c-446a-8fbd-68e4cd3c1a53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"
Aug 05 17:41:11.126979 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.126977    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljt4s\" (UniqueName: \"kubernetes.io/projected/23c743b9-c19c-446a-8fbd-68e4cd3c1a53-kube-api-access-ljt4s\") pod \"kuadrant-operator-controller-manager-6bd956ccf-w926d\" (UID: \"23c743b9-c19c-446a-8fbd-68e4cd3c1a53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"
Aug 05 17:41:11.172805 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.172774    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"]
Aug 05 17:41:11.173073 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:41:11.173054    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-ljt4s], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d" podUID="23c743b9-c19c-446a-8fbd-68e4cd3c1a53"
Aug 05 17:41:11.227584 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.227543    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23c743b9-c19c-446a-8fbd-68e4cd3c1a53-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-w926d\" (UID: \"23c743b9-c19c-446a-8fbd-68e4cd3c1a53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"
Aug 05 17:41:11.227584 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.227589    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljt4s\" (UniqueName: \"kubernetes.io/projected/23c743b9-c19c-446a-8fbd-68e4cd3c1a53-kube-api-access-ljt4s\") pod \"kuadrant-operator-controller-manager-6bd956ccf-w926d\" (UID: \"23c743b9-c19c-446a-8fbd-68e4cd3c1a53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"
Aug 05 17:41:11.227959 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.227941    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23c743b9-c19c-446a-8fbd-68e4cd3c1a53-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-w926d\" (UID: \"23c743b9-c19c-446a-8fbd-68e4cd3c1a53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"
Aug 05 17:41:11.238319 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.238288    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljt4s\" (UniqueName: \"kubernetes.io/projected/23c743b9-c19c-446a-8fbd-68e4cd3c1a53-kube-api-access-ljt4s\") pod \"kuadrant-operator-controller-manager-6bd956ccf-w926d\" (UID: \"23c743b9-c19c-446a-8fbd-68e4cd3c1a53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"
Aug 05 17:41:11.305072 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.305038    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"]
Aug 05 17:41:11.320944 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.320916    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"]
Aug 05 17:41:11.322957 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.322929    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:11.331475 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.331447    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx"]
Aug 05 17:41:11.335493 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.335468    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx"
Aug 05 17:41:11.353765 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.353739    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx"]
Aug 05 17:41:11.412164 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.412131    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:11.429591 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.429561    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwtf5\" (UniqueName: \"kubernetes.io/projected/e5d029d0-2470-4054-8ef9-4d3eb9aaf600-kube-api-access-hwtf5\") pod \"kuadrant-operator-controller-manager-68cf589589-2j6wx\" (UID: \"e5d029d0-2470-4054-8ef9-4d3eb9aaf600\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx"
Aug 05 17:41:11.429736 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.429635    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5d029d0-2470-4054-8ef9-4d3eb9aaf600-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-2j6wx\" (UID: \"e5d029d0-2470-4054-8ef9-4d3eb9aaf600\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx"
Aug 05 17:41:11.441341 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.441317    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"
Aug 05 17:41:11.443992 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:41:11.443963    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-w926d\" is forbidden: User \"system:node:ip-10-0-131-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-77.ec2.internal' and this object" podUID="23c743b9-c19c-446a-8fbd-68e4cd3c1a53" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"
Aug 05 17:41:11.447814 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.447797    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"
Aug 05 17:41:11.450246 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:41:11.450223    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-w926d\" is forbidden: User \"system:node:ip-10-0-131-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-77.ec2.internal' and this object" podUID="23c743b9-c19c-446a-8fbd-68e4cd3c1a53" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"
Aug 05 17:41:11.530603 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.530574    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23c743b9-c19c-446a-8fbd-68e4cd3c1a53-extensions-socket-volume\") pod \"23c743b9-c19c-446a-8fbd-68e4cd3c1a53\" (UID: \"23c743b9-c19c-446a-8fbd-68e4cd3c1a53\") "
Aug 05 17:41:11.530753 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.530652    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljt4s\" (UniqueName: \"kubernetes.io/projected/23c743b9-c19c-446a-8fbd-68e4cd3c1a53-kube-api-access-ljt4s\") pod \"23c743b9-c19c-446a-8fbd-68e4cd3c1a53\" (UID: \"23c743b9-c19c-446a-8fbd-68e4cd3c1a53\") "
Aug 05 17:41:11.530753 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.530733    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwtf5\" (UniqueName: \"kubernetes.io/projected/e5d029d0-2470-4054-8ef9-4d3eb9aaf600-kube-api-access-hwtf5\") pod \"kuadrant-operator-controller-manager-68cf589589-2j6wx\" (UID: \"e5d029d0-2470-4054-8ef9-4d3eb9aaf600\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx"
Aug 05 17:41:11.530859 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.530785    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5d029d0-2470-4054-8ef9-4d3eb9aaf600-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-2j6wx\" (UID: \"e5d029d0-2470-4054-8ef9-4d3eb9aaf600\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx"
Aug 05 17:41:11.530859 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.530813    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23c743b9-c19c-446a-8fbd-68e4cd3c1a53-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "23c743b9-c19c-446a-8fbd-68e4cd3c1a53" (UID: "23c743b9-c19c-446a-8fbd-68e4cd3c1a53"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:41:11.531077 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.531055    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5d029d0-2470-4054-8ef9-4d3eb9aaf600-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-2j6wx\" (UID: \"e5d029d0-2470-4054-8ef9-4d3eb9aaf600\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx"
Aug 05 17:41:11.532941 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.532920    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23c743b9-c19c-446a-8fbd-68e4cd3c1a53-kube-api-access-ljt4s" (OuterVolumeSpecName: "kube-api-access-ljt4s") pod "23c743b9-c19c-446a-8fbd-68e4cd3c1a53" (UID: "23c743b9-c19c-446a-8fbd-68e4cd3c1a53"). InnerVolumeSpecName "kube-api-access-ljt4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:41:11.539766 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.539743    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwtf5\" (UniqueName: \"kubernetes.io/projected/e5d029d0-2470-4054-8ef9-4d3eb9aaf600-kube-api-access-hwtf5\") pod \"kuadrant-operator-controller-manager-68cf589589-2j6wx\" (UID: \"e5d029d0-2470-4054-8ef9-4d3eb9aaf600\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx"
Aug 05 17:41:11.631583 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.631511    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljt4s\" (UniqueName: \"kubernetes.io/projected/23c743b9-c19c-446a-8fbd-68e4cd3c1a53-kube-api-access-ljt4s\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:41:11.631583 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.631536    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23c743b9-c19c-446a-8fbd-68e4cd3c1a53-extensions-socket-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:41:11.649439 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.649414    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx"
Aug 05 17:41:11.785805 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:11.785783    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx"]
Aug 05 17:41:11.788285 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:41:11.788261    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5d029d0_2470_4054_8ef9_4d3eb9aaf600.slice/crio-4287b37449e4ec12951ad061028d544b1cab30d5d2f0fb5e01592fede9b88212 WatchSource:0}: Error finding container 4287b37449e4ec12951ad061028d544b1cab30d5d2f0fb5e01592fede9b88212: Status 404 returned error can't find the container with id 4287b37449e4ec12951ad061028d544b1cab30d5d2f0fb5e01592fede9b88212
Aug 05 17:41:12.449112 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:12.449088    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"
Aug 05 17:41:12.449630 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:12.449123    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx" event={"ID":"e5d029d0-2470-4054-8ef9-4d3eb9aaf600","Type":"ContainerStarted","Data":"4287b37449e4ec12951ad061028d544b1cab30d5d2f0fb5e01592fede9b88212"}
Aug 05 17:41:12.451723 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:41:12.451690    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-w926d\" is forbidden: User \"system:node:ip-10-0-131-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-77.ec2.internal' and this object" podUID="23c743b9-c19c-446a-8fbd-68e4cd3c1a53" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"
Aug 05 17:41:12.459218 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:41:12.459190    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-w926d\" is forbidden: User \"system:node:ip-10-0-131-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-77.ec2.internal' and this object" podUID="23c743b9-c19c-446a-8fbd-68e4cd3c1a53" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"
Aug 05 17:41:13.150405 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:41:13.150367    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-w926d\" is forbidden: User \"system:node:ip-10-0-131-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-77.ec2.internal' and this object" podUID="23c743b9-c19c-446a-8fbd-68e4cd3c1a53" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-w926d"
Aug 05 17:41:13.152086 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:13.152062    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23c743b9-c19c-446a-8fbd-68e4cd3c1a53" path="/var/lib/kubelet/pods/23c743b9-c19c-446a-8fbd-68e4cd3c1a53/volumes"
Aug 05 17:41:15.465963 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:15.465925    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx" event={"ID":"e5d029d0-2470-4054-8ef9-4d3eb9aaf600","Type":"ContainerStarted","Data":"d682cad31201b572f3bf73c30ed7de01fd770cdb14f62331ce79ec5634b48dc3"}
Aug 05 17:41:15.466345 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:15.466062    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx"
Aug 05 17:41:15.492054 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:15.491998    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx" podStartSLOduration=0.986725191 podStartE2EDuration="4.491984678s" totalImagesPullingTime="3.505259487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:41:15.490154879 +0000 UTC m=+482.842829473" watchObservedRunningTime="2026-08-05 17:41:15.491984678 +0000 UTC m=+482.844659271"
Aug 05 17:41:15.888767 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:15.888724    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:15.988685 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:15.988647    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:16.521517 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:16.521486    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:16.618338 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:16.618304    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:18.900727 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:18.900696    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:19.001545 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:19.001521    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:19.773812 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:19.773787    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:19.876643 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:19.876616    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:24.095371 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:24.095334    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:24.197449 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:24.197420    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:24.304464 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:24.304438    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:24.409455 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:24.409382    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:24.508190 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:24.508160    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:24.606110 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:24.606074    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:26.472820 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:26.472786    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx"
Aug 05 17:41:26.858858 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:26.858825    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:30.257935 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:30.257896    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:30.359161 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:30.359124    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:43.313139 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:43.313103    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:43.410649 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:43.410616    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:44.873039 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:44.873001    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:44.958190 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:44.958117    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:45.028604 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:45.028570    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:45.652081 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:45.652056    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-pnjsm"]
Aug 05 17:41:45.677493 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:45.677463    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-pnjsm"]
Aug 05 17:41:45.677683 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:45.677569    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-pnjsm"
Aug 05 17:41:45.681103 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:45.681081    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-99fvc\""
Aug 05 17:41:45.688829 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:45.688809    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:45.752820 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:45.752790    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:45.829784 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:45.829753    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gz2d\" (UniqueName: \"kubernetes.io/projected/8dfaa36c-3243-4968-a8f3-525831e26493-kube-api-access-5gz2d\") pod \"authorino-b54df94b4-pnjsm\" (UID: \"8dfaa36c-3243-4968-a8f3-525831e26493\") " pod="kuadrant-system/authorino-b54df94b4-pnjsm"
Aug 05 17:41:45.930279 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:45.930204    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gz2d\" (UniqueName: \"kubernetes.io/projected/8dfaa36c-3243-4968-a8f3-525831e26493-kube-api-access-5gz2d\") pod \"authorino-b54df94b4-pnjsm\" (UID: \"8dfaa36c-3243-4968-a8f3-525831e26493\") " pod="kuadrant-system/authorino-b54df94b4-pnjsm"
Aug 05 17:41:45.939349 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:45.939330    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gz2d\" (UniqueName: \"kubernetes.io/projected/8dfaa36c-3243-4968-a8f3-525831e26493-kube-api-access-5gz2d\") pod \"authorino-b54df94b4-pnjsm\" (UID: \"8dfaa36c-3243-4968-a8f3-525831e26493\") " pod="kuadrant-system/authorino-b54df94b4-pnjsm"
Aug 05 17:41:45.987418 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:45.987391    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-pnjsm"
Aug 05 17:41:46.139688 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:46.139665    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-pnjsm"]
Aug 05 17:41:46.142239 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:41:46.142211    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8dfaa36c_3243_4968_a8f3_525831e26493.slice/crio-ef45cba2789a163e74e37a3dcfe7ad5934779381ea95445befa934b38c380814 WatchSource:0}: Error finding container ef45cba2789a163e74e37a3dcfe7ad5934779381ea95445befa934b38c380814: Status 404 returned error can't find the container with id ef45cba2789a163e74e37a3dcfe7ad5934779381ea95445befa934b38c380814
Aug 05 17:41:46.610670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:46.610639    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-pnjsm" event={"ID":"8dfaa36c-3243-4968-a8f3-525831e26493","Type":"ContainerStarted","Data":"ef45cba2789a163e74e37a3dcfe7ad5934779381ea95445befa934b38c380814"}
Aug 05 17:41:50.633042 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:50.633007    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-pnjsm" event={"ID":"8dfaa36c-3243-4968-a8f3-525831e26493","Type":"ContainerStarted","Data":"7b5cf2ee18fa7c1a64f5ef14cdaad7d951889afe8de520cb516c8af2e1c2d416"}
Aug 05 17:41:50.649898 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:50.649856    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-pnjsm" podStartSLOduration=1.365607999 podStartE2EDuration="5.649841871s" totalImagesPullingTime="4.284233872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:41:50.648895841 +0000 UTC m=+518.001570437" watchObservedRunningTime="2026-08-05 17:41:50.649841871 +0000 UTC m=+518.002516464"
Aug 05 17:41:50.963130 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:50.963092    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-pnjsm"]
Aug 05 17:41:52.642252 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:52.642149    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-pnjsm" podUID="8dfaa36c-3243-4968-a8f3-525831e26493" containerName="authorino" containerID="cri-o://7b5cf2ee18fa7c1a64f5ef14cdaad7d951889afe8de520cb516c8af2e1c2d416" gracePeriod=30
Aug 05 17:41:52.885890 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:52.885864    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-pnjsm"
Aug 05 17:41:52.998414 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:52.998353    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gz2d\" (UniqueName: \"kubernetes.io/projected/8dfaa36c-3243-4968-a8f3-525831e26493-kube-api-access-5gz2d\") pod \"8dfaa36c-3243-4968-a8f3-525831e26493\" (UID: \"8dfaa36c-3243-4968-a8f3-525831e26493\") "
Aug 05 17:41:53.000662 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:53.000635    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dfaa36c-3243-4968-a8f3-525831e26493-kube-api-access-5gz2d" (OuterVolumeSpecName: "kube-api-access-5gz2d") pod "8dfaa36c-3243-4968-a8f3-525831e26493" (UID: "8dfaa36c-3243-4968-a8f3-525831e26493"). InnerVolumeSpecName "kube-api-access-5gz2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:41:53.099011 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:53.098990    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gz2d\" (UniqueName: \"kubernetes.io/projected/8dfaa36c-3243-4968-a8f3-525831e26493-kube-api-access-5gz2d\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:41:53.648625 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:53.648588    2569 generic.go:356] "Generic (PLEG): container finished" podID="8dfaa36c-3243-4968-a8f3-525831e26493" containerID="7b5cf2ee18fa7c1a64f5ef14cdaad7d951889afe8de520cb516c8af2e1c2d416" exitCode=0
Aug 05 17:41:53.649039 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:53.648641    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-pnjsm" event={"ID":"8dfaa36c-3243-4968-a8f3-525831e26493","Type":"ContainerDied","Data":"7b5cf2ee18fa7c1a64f5ef14cdaad7d951889afe8de520cb516c8af2e1c2d416"}
Aug 05 17:41:53.649039 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:53.648653    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-pnjsm"
Aug 05 17:41:53.649039 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:53.648676    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-pnjsm" event={"ID":"8dfaa36c-3243-4968-a8f3-525831e26493","Type":"ContainerDied","Data":"ef45cba2789a163e74e37a3dcfe7ad5934779381ea95445befa934b38c380814"}
Aug 05 17:41:53.649039 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:53.648696    2569 scope.go:117] "RemoveContainer" containerID="7b5cf2ee18fa7c1a64f5ef14cdaad7d951889afe8de520cb516c8af2e1c2d416"
Aug 05 17:41:53.657452 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:53.657432    2569 scope.go:117] "RemoveContainer" containerID="7b5cf2ee18fa7c1a64f5ef14cdaad7d951889afe8de520cb516c8af2e1c2d416"
Aug 05 17:41:53.657751 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:41:53.657729    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b5cf2ee18fa7c1a64f5ef14cdaad7d951889afe8de520cb516c8af2e1c2d416\": container with ID starting with 7b5cf2ee18fa7c1a64f5ef14cdaad7d951889afe8de520cb516c8af2e1c2d416 not found: ID does not exist" containerID="7b5cf2ee18fa7c1a64f5ef14cdaad7d951889afe8de520cb516c8af2e1c2d416"
Aug 05 17:41:53.657797 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:53.657765    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b5cf2ee18fa7c1a64f5ef14cdaad7d951889afe8de520cb516c8af2e1c2d416"} err="failed to get container status \"7b5cf2ee18fa7c1a64f5ef14cdaad7d951889afe8de520cb516c8af2e1c2d416\": rpc error: code = NotFound desc = could not find container \"7b5cf2ee18fa7c1a64f5ef14cdaad7d951889afe8de520cb516c8af2e1c2d416\": container with ID starting with 7b5cf2ee18fa7c1a64f5ef14cdaad7d951889afe8de520cb516c8af2e1c2d416 not found: ID does not exist"
Aug 05 17:41:53.665679 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:53.665654    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-pnjsm"]
Aug 05 17:41:53.668331 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:53.668309    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-pnjsm"]
Aug 05 17:41:55.150054 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:55.150021    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dfaa36c-3243-4968-a8f3-525831e26493" path="/var/lib/kubelet/pods/8dfaa36c-3243-4968-a8f3-525831e26493/volumes"
Aug 05 17:41:57.811339 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:41:57.811310    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:06.062365 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:06.062287    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:06.160417 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:06.160390    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:06.477422 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:06.477393    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:13.786799 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:13.786766    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:13.842496 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:13.842455    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:18.389905 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:18.389878    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:18.610350 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:18.610313    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:24.611746 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:24.611714    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:24.680208 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:24.680161    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:24.739912 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:24.739887    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:24.773761 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:24.773734    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67657dfbc-jc99s"]
Aug 05 17:42:24.774199 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:24.774164    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dfaa36c-3243-4968-a8f3-525831e26493" containerName="authorino"
Aug 05 17:42:24.774199 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:24.774197    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dfaa36c-3243-4968-a8f3-525831e26493" containerName="authorino"
Aug 05 17:42:24.774389 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:24.774375    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dfaa36c-3243-4968-a8f3-525831e26493" containerName="authorino"
Aug 05 17:42:24.780257 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:24.780240    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67657dfbc-jc99s"
Aug 05 17:42:24.783795 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:24.783779    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 05 17:42:24.783895 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:24.783777    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ssqzs\""
Aug 05 17:42:24.783895 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:24.783779    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 05 17:42:24.789731 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:24.789702    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67657dfbc-jc99s"]
Aug 05 17:42:24.854949 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:24.854922    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7s8m\" (UniqueName: \"kubernetes.io/projected/de9f555f-7afc-4a8a-be1e-1ff50e0efd5e-kube-api-access-v7s8m\") pod \"maas-controller-67657dfbc-jc99s\" (UID: \"de9f555f-7afc-4a8a-be1e-1ff50e0efd5e\") " pod="opendatahub/maas-controller-67657dfbc-jc99s"
Aug 05 17:42:24.855060 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:24.854975    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de9f555f-7afc-4a8a-be1e-1ff50e0efd5e-webhook-cert\") pod \"maas-controller-67657dfbc-jc99s\" (UID: \"de9f555f-7afc-4a8a-be1e-1ff50e0efd5e\") " pod="opendatahub/maas-controller-67657dfbc-jc99s"
Aug 05 17:42:24.955833 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:24.955777    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7s8m\" (UniqueName: \"kubernetes.io/projected/de9f555f-7afc-4a8a-be1e-1ff50e0efd5e-kube-api-access-v7s8m\") pod \"maas-controller-67657dfbc-jc99s\" (UID: \"de9f555f-7afc-4a8a-be1e-1ff50e0efd5e\") " pod="opendatahub/maas-controller-67657dfbc-jc99s"
Aug 05 17:42:24.955833 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:24.955821    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de9f555f-7afc-4a8a-be1e-1ff50e0efd5e-webhook-cert\") pod \"maas-controller-67657dfbc-jc99s\" (UID: \"de9f555f-7afc-4a8a-be1e-1ff50e0efd5e\") " pod="opendatahub/maas-controller-67657dfbc-jc99s"
Aug 05 17:42:24.958363 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:24.958337    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de9f555f-7afc-4a8a-be1e-1ff50e0efd5e-webhook-cert\") pod \"maas-controller-67657dfbc-jc99s\" (UID: \"de9f555f-7afc-4a8a-be1e-1ff50e0efd5e\") " pod="opendatahub/maas-controller-67657dfbc-jc99s"
Aug 05 17:42:24.963808 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:24.963787    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7s8m\" (UniqueName: \"kubernetes.io/projected/de9f555f-7afc-4a8a-be1e-1ff50e0efd5e-kube-api-access-v7s8m\") pod \"maas-controller-67657dfbc-jc99s\" (UID: \"de9f555f-7afc-4a8a-be1e-1ff50e0efd5e\") " pod="opendatahub/maas-controller-67657dfbc-jc99s"
Aug 05 17:42:25.091443 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:25.091421    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67657dfbc-jc99s"
Aug 05 17:42:25.223286 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:25.223264    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67657dfbc-jc99s"]
Aug 05 17:42:25.225335 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:42:25.225306    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde9f555f_7afc_4a8a_be1e_1ff50e0efd5e.slice/crio-9ecebafd5c241d0d12b7156ee9d20ce3bdebcbac5fb86cfd41bcc7db428bb7c3 WatchSource:0}: Error finding container 9ecebafd5c241d0d12b7156ee9d20ce3bdebcbac5fb86cfd41bcc7db428bb7c3: Status 404 returned error can't find the container with id 9ecebafd5c241d0d12b7156ee9d20ce3bdebcbac5fb86cfd41bcc7db428bb7c3
Aug 05 17:42:25.812173 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:25.812135    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67657dfbc-jc99s" event={"ID":"de9f555f-7afc-4a8a-be1e-1ff50e0efd5e","Type":"ContainerStarted","Data":"9ecebafd5c241d0d12b7156ee9d20ce3bdebcbac5fb86cfd41bcc7db428bb7c3"}
Aug 05 17:42:27.621824 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:27.621797    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:27.718040 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:27.717968    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:27.825046 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:27.825012    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67657dfbc-jc99s" event={"ID":"de9f555f-7afc-4a8a-be1e-1ff50e0efd5e","Type":"ContainerStarted","Data":"31e59b5395ee3a8f56710a900304073ae96e9945c575c39de57297e6148fa450"}
Aug 05 17:42:27.825251 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:27.825136    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67657dfbc-jc99s"
Aug 05 17:42:27.839924 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:27.839897    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:27.846247 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:27.846209    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67657dfbc-jc99s" podStartSLOduration=1.605945739 podStartE2EDuration="3.846194842s" totalImagesPullingTime="2.240249103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:42:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:42:27.844677956 +0000 UTC m=+555.197352553" watchObservedRunningTime="2026-08-05 17:42:27.846194842 +0000 UTC m=+555.198869427"
Aug 05 17:42:38.619536 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:38.619506    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:38.832809 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:38.832783    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67657dfbc-jc99s"
Aug 05 17:42:59.254398 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:59.254355    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:59.353833 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:42:59.353762    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:00.846463 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:00.846437    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:00.885022 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:00.884987    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5888db464-dll25"]
Aug 05 17:43:00.893740 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:00.893716    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5888db464-dll25"
Aug 05 17:43:00.896718 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:00.896696    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 05 17:43:00.896718 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:00.896715    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Aug 05 17:43:00.896873 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:00.896705    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Aug 05 17:43:00.900727 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:00.900704    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5888db464-dll25"]
Aug 05 17:43:01.066685 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.066646    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c26998e-fd41-41dd-9530-0b7139f5372a-maas-api-tls\") pod \"maas-api-5888db464-dll25\" (UID: \"8c26998e-fd41-41dd-9530-0b7139f5372a\") " pod="odh-ai-gateway-infra/maas-api-5888db464-dll25"
Aug 05 17:43:01.066685 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.066690    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vlpw\" (UniqueName: \"kubernetes.io/projected/8c26998e-fd41-41dd-9530-0b7139f5372a-kube-api-access-4vlpw\") pod \"maas-api-5888db464-dll25\" (UID: \"8c26998e-fd41-41dd-9530-0b7139f5372a\") " pod="odh-ai-gateway-infra/maas-api-5888db464-dll25"
Aug 05 17:43:01.066903 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.066792    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c26998e-fd41-41dd-9530-0b7139f5372a-openshift-service-ca\") pod \"maas-api-5888db464-dll25\" (UID: \"8c26998e-fd41-41dd-9530-0b7139f5372a\") " pod="odh-ai-gateway-infra/maas-api-5888db464-dll25"
Aug 05 17:43:01.088865 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.088830    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:01.167800 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.167713    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c26998e-fd41-41dd-9530-0b7139f5372a-openshift-service-ca\") pod \"maas-api-5888db464-dll25\" (UID: \"8c26998e-fd41-41dd-9530-0b7139f5372a\") " pod="odh-ai-gateway-infra/maas-api-5888db464-dll25"
Aug 05 17:43:01.167800 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.167791    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c26998e-fd41-41dd-9530-0b7139f5372a-maas-api-tls\") pod \"maas-api-5888db464-dll25\" (UID: \"8c26998e-fd41-41dd-9530-0b7139f5372a\") " pod="odh-ai-gateway-infra/maas-api-5888db464-dll25"
Aug 05 17:43:01.168025 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.167826    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vlpw\" (UniqueName: \"kubernetes.io/projected/8c26998e-fd41-41dd-9530-0b7139f5372a-kube-api-access-4vlpw\") pod \"maas-api-5888db464-dll25\" (UID: \"8c26998e-fd41-41dd-9530-0b7139f5372a\") " pod="odh-ai-gateway-infra/maas-api-5888db464-dll25"
Aug 05 17:43:01.168401 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.168378    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c26998e-fd41-41dd-9530-0b7139f5372a-openshift-service-ca\") pod \"maas-api-5888db464-dll25\" (UID: \"8c26998e-fd41-41dd-9530-0b7139f5372a\") " pod="odh-ai-gateway-infra/maas-api-5888db464-dll25"
Aug 05 17:43:01.170525 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.170506    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c26998e-fd41-41dd-9530-0b7139f5372a-maas-api-tls\") pod \"maas-api-5888db464-dll25\" (UID: \"8c26998e-fd41-41dd-9530-0b7139f5372a\") " pod="odh-ai-gateway-infra/maas-api-5888db464-dll25"
Aug 05 17:43:01.176270 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.176242    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vlpw\" (UniqueName: \"kubernetes.io/projected/8c26998e-fd41-41dd-9530-0b7139f5372a-kube-api-access-4vlpw\") pod \"maas-api-5888db464-dll25\" (UID: \"8c26998e-fd41-41dd-9530-0b7139f5372a\") " pod="odh-ai-gateway-infra/maas-api-5888db464-dll25"
Aug 05 17:43:01.205114 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.205075    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5888db464-dll25"
Aug 05 17:43:01.335878 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.335820    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:01.360454 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.360424    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5888db464-dll25"]
Aug 05 17:43:01.362863 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:43:01.362825    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c26998e_fd41_41dd_9530_0b7139f5372a.slice/crio-838faae67c189b6154d2ed73f9e2d05bbb70c5636e698669f97349573ccefdc7 WatchSource:0}: Error finding container 838faae67c189b6154d2ed73f9e2d05bbb70c5636e698669f97349573ccefdc7: Status 404 returned error can't find the container with id 838faae67c189b6154d2ed73f9e2d05bbb70c5636e698669f97349573ccefdc7
Aug 05 17:43:01.432951 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.432844    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:01.433397 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.433374    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-5phph"]
Aug 05 17:43:01.438635 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.438610    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-5phph"
Aug 05 17:43:01.441960 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.441931    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Aug 05 17:43:01.448233 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.448204    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-5phph"]
Aug 05 17:43:01.459015 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.458988    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-6tnnr"]
Aug 05 17:43:01.462907 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.462882    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-6tnnr"
Aug 05 17:43:01.465487 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.465450    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-cmkh6\""
Aug 05 17:43:01.471500 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.470821    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57db4d70-23ff-467d-a092-477df5cc8365-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-5phph\" (UID: \"57db4d70-23ff-467d-a092-477df5cc8365\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-5phph"
Aug 05 17:43:01.471500 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.470882    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b205b96-927c-4170-a375-5190000f495a-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-6tnnr\" (UID: \"7b205b96-927c-4170-a375-5190000f495a\") " pod="openshift-ingress/payload-processing-5cc6cd444b-6tnnr"
Aug 05 17:43:01.471500 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.470911    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzd7v\" (UniqueName: \"kubernetes.io/projected/57db4d70-23ff-467d-a092-477df5cc8365-kube-api-access-vzd7v\") pod \"payload-pre-processing-687dbb6cf6-5phph\" (UID: \"57db4d70-23ff-467d-a092-477df5cc8365\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-5phph"
Aug 05 17:43:01.471500 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.471071    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dkrs\" (UniqueName: \"kubernetes.io/projected/7b205b96-927c-4170-a375-5190000f495a-kube-api-access-6dkrs\") pod \"payload-processing-5cc6cd444b-6tnnr\" (UID: \"7b205b96-927c-4170-a375-5190000f495a\") " pod="openshift-ingress/payload-processing-5cc6cd444b-6tnnr"
Aug 05 17:43:01.474648 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.474619    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-6tnnr"]
Aug 05 17:43:01.572258 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.572226    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dkrs\" (UniqueName: \"kubernetes.io/projected/7b205b96-927c-4170-a375-5190000f495a-kube-api-access-6dkrs\") pod \"payload-processing-5cc6cd444b-6tnnr\" (UID: \"7b205b96-927c-4170-a375-5190000f495a\") " pod="openshift-ingress/payload-processing-5cc6cd444b-6tnnr"
Aug 05 17:43:01.572467 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.572285    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57db4d70-23ff-467d-a092-477df5cc8365-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-5phph\" (UID: \"57db4d70-23ff-467d-a092-477df5cc8365\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-5phph"
Aug 05 17:43:01.572467 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.572326    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b205b96-927c-4170-a375-5190000f495a-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-6tnnr\" (UID: \"7b205b96-927c-4170-a375-5190000f495a\") " pod="openshift-ingress/payload-processing-5cc6cd444b-6tnnr"
Aug 05 17:43:01.572467 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.572349    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzd7v\" (UniqueName: \"kubernetes.io/projected/57db4d70-23ff-467d-a092-477df5cc8365-kube-api-access-vzd7v\") pod \"payload-pre-processing-687dbb6cf6-5phph\" (UID: \"57db4d70-23ff-467d-a092-477df5cc8365\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-5phph"
Aug 05 17:43:01.573231 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.573204    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57db4d70-23ff-467d-a092-477df5cc8365-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-5phph\" (UID: \"57db4d70-23ff-467d-a092-477df5cc8365\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-5phph"
Aug 05 17:43:01.573291 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.573251    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b205b96-927c-4170-a375-5190000f495a-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-6tnnr\" (UID: \"7b205b96-927c-4170-a375-5190000f495a\") " pod="openshift-ingress/payload-processing-5cc6cd444b-6tnnr"
Aug 05 17:43:01.581560 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.581529    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dkrs\" (UniqueName: \"kubernetes.io/projected/7b205b96-927c-4170-a375-5190000f495a-kube-api-access-6dkrs\") pod \"payload-processing-5cc6cd444b-6tnnr\" (UID: \"7b205b96-927c-4170-a375-5190000f495a\") " pod="openshift-ingress/payload-processing-5cc6cd444b-6tnnr"
Aug 05 17:43:01.582097 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.582070    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzd7v\" (UniqueName: \"kubernetes.io/projected/57db4d70-23ff-467d-a092-477df5cc8365-kube-api-access-vzd7v\") pod \"payload-pre-processing-687dbb6cf6-5phph\" (UID: \"57db4d70-23ff-467d-a092-477df5cc8365\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-5phph"
Aug 05 17:43:01.752465 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.752374    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-5phph"
Aug 05 17:43:01.776234 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.776202    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-6tnnr"
Aug 05 17:43:01.913357 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.913323    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-5phph"]
Aug 05 17:43:01.914940 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:43:01.914912    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57db4d70_23ff_467d_a092_477df5cc8365.slice/crio-2715ac96b0408dc6849761771d8881ca1dc24bca3a6763fe33e7aa4e0cca80fa WatchSource:0}: Error finding container 2715ac96b0408dc6849761771d8881ca1dc24bca3a6763fe33e7aa4e0cca80fa: Status 404 returned error can't find the container with id 2715ac96b0408dc6849761771d8881ca1dc24bca3a6763fe33e7aa4e0cca80fa
Aug 05 17:43:01.942581 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:01.942542    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-6tnnr"]
Aug 05 17:43:01.945515 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:43:01.945491    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b205b96_927c_4170_a375_5190000f495a.slice/crio-8d5ef6aa4c1a50b9f3d15d4c9efd6bd5645b639e318040452413bd4921e8fa59 WatchSource:0}: Error finding container 8d5ef6aa4c1a50b9f3d15d4c9efd6bd5645b639e318040452413bd4921e8fa59: Status 404 returned error can't find the container with id 8d5ef6aa4c1a50b9f3d15d4c9efd6bd5645b639e318040452413bd4921e8fa59
Aug 05 17:43:02.001097 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:02.001050    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-6tnnr" event={"ID":"7b205b96-927c-4170-a375-5190000f495a","Type":"ContainerStarted","Data":"8d5ef6aa4c1a50b9f3d15d4c9efd6bd5645b639e318040452413bd4921e8fa59"}
Aug 05 17:43:02.002443 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:02.002386    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-5phph" event={"ID":"57db4d70-23ff-467d-a092-477df5cc8365","Type":"ContainerStarted","Data":"2715ac96b0408dc6849761771d8881ca1dc24bca3a6763fe33e7aa4e0cca80fa"}
Aug 05 17:43:02.003674 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:02.003607    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5888db464-dll25" event={"ID":"8c26998e-fd41-41dd-9530-0b7139f5372a","Type":"ContainerStarted","Data":"838faae67c189b6154d2ed73f9e2d05bbb70c5636e698669f97349573ccefdc7"}
Aug 05 17:43:04.022994 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:04.022908    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-6tnnr" event={"ID":"7b205b96-927c-4170-a375-5190000f495a","Type":"ContainerStarted","Data":"9c41432aa7178a9698a523b1c4f082823e8441132203b663fc2c8629dfb26de6"}
Aug 05 17:43:04.022994 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:04.022982    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-6tnnr"
Aug 05 17:43:04.024434 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:04.024410    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-5phph" event={"ID":"57db4d70-23ff-467d-a092-477df5cc8365","Type":"ContainerStarted","Data":"ae7d316f10077bc1472d83cf2132fd55cbadb479d5b2443c14d182d979bbccac"}
Aug 05 17:43:04.024550 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:04.024491    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-5phph"
Aug 05 17:43:04.025898 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:04.025876    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5888db464-dll25" event={"ID":"8c26998e-fd41-41dd-9530-0b7139f5372a","Type":"ContainerStarted","Data":"e59a21af6624356b21e5e3474b60fd5e767d4af7aa2c37a6ecb27b9a8baf94bf"}
Aug 05 17:43:04.026010 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:04.025987    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5888db464-dll25"
Aug 05 17:43:04.043693 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:04.043658    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-6tnnr" podStartSLOduration=1.20817961 podStartE2EDuration="3.04364554s" totalImagesPullingTime="1.83546593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:43:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:43:04.041317816 +0000 UTC m=+591.393992410" watchObservedRunningTime="2026-08-05 17:43:04.04364554 +0000 UTC m=+591.396320133"
Aug 05 17:43:04.057228 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:04.057190    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-5phph" podStartSLOduration=1.197679076 podStartE2EDuration="3.057162351s" totalImagesPullingTime="1.859483275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:43:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:43:04.055911913 +0000 UTC m=+591.408586539" watchObservedRunningTime="2026-08-05 17:43:04.057162351 +0000 UTC m=+591.409836944"
Aug 05 17:43:04.072426 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:04.072391    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5888db464-dll25" podStartSLOduration=2.040279919 podStartE2EDuration="4.072378637s" totalImagesPullingTime="2.032098718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:43:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:43:04.071844102 +0000 UTC m=+591.424518698" watchObservedRunningTime="2026-08-05 17:43:04.072378637 +0000 UTC m=+591.425053231"
Aug 05 17:43:10.036149 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:10.036113    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5888db464-dll25"
Aug 05 17:43:11.000815 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:11.000785    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:11.098003 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:11.097973    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:12.672365 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:12.672336    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:14.912078 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:14.912047    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:15.033387 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:15.033358    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-5phph"
Aug 05 17:43:15.033564 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:15.033433    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-6tnnr"
Aug 05 17:43:18.066301 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:18.066262    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:26.530932 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:26.530848    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:45.765260 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:45.765217    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:55.721995 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.721962    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"]
Aug 05 17:43:55.727043 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.727019    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:55.730727 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.730699    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-274cz\""
Aug 05 17:43:55.730850 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.730719    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Aug 05 17:43:55.730850 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.730719    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Aug 05 17:43:55.730850 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.730808    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Aug 05 17:43:55.737856 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.737816    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"]
Aug 05 17:43:55.751322 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.751300    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:55.831059 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.831037    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6417b863-e45b-460b-9e49-1aae53077cd5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9\" (UID: \"6417b863-e45b-460b-9e49-1aae53077cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:55.831219 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.831071    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6417b863-e45b-460b-9e49-1aae53077cd5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9\" (UID: \"6417b863-e45b-460b-9e49-1aae53077cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:55.831219 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.831107    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6417b863-e45b-460b-9e49-1aae53077cd5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9\" (UID: \"6417b863-e45b-460b-9e49-1aae53077cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:55.831219 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.831204    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpltq\" (UniqueName: \"kubernetes.io/projected/6417b863-e45b-460b-9e49-1aae53077cd5-kube-api-access-dpltq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9\" (UID: \"6417b863-e45b-460b-9e49-1aae53077cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:55.831440 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.831263    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6417b863-e45b-460b-9e49-1aae53077cd5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9\" (UID: \"6417b863-e45b-460b-9e49-1aae53077cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:55.831440 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.831290    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6417b863-e45b-460b-9e49-1aae53077cd5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9\" (UID: \"6417b863-e45b-460b-9e49-1aae53077cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:55.932279 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.932255    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6417b863-e45b-460b-9e49-1aae53077cd5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9\" (UID: \"6417b863-e45b-460b-9e49-1aae53077cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:55.932378 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.932282    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6417b863-e45b-460b-9e49-1aae53077cd5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9\" (UID: \"6417b863-e45b-460b-9e49-1aae53077cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:55.932378 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.932309    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6417b863-e45b-460b-9e49-1aae53077cd5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9\" (UID: \"6417b863-e45b-460b-9e49-1aae53077cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:55.932378 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.932362    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpltq\" (UniqueName: \"kubernetes.io/projected/6417b863-e45b-460b-9e49-1aae53077cd5-kube-api-access-dpltq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9\" (UID: \"6417b863-e45b-460b-9e49-1aae53077cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:55.932513 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.932409    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6417b863-e45b-460b-9e49-1aae53077cd5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9\" (UID: \"6417b863-e45b-460b-9e49-1aae53077cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:55.932513 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.932432    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6417b863-e45b-460b-9e49-1aae53077cd5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9\" (UID: \"6417b863-e45b-460b-9e49-1aae53077cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:55.932697 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.932677    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6417b863-e45b-460b-9e49-1aae53077cd5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9\" (UID: \"6417b863-e45b-460b-9e49-1aae53077cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:55.932771 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.932747    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6417b863-e45b-460b-9e49-1aae53077cd5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9\" (UID: \"6417b863-e45b-460b-9e49-1aae53077cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:55.932832 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.932795    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6417b863-e45b-460b-9e49-1aae53077cd5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9\" (UID: \"6417b863-e45b-460b-9e49-1aae53077cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:55.934591 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.934575    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6417b863-e45b-460b-9e49-1aae53077cd5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9\" (UID: \"6417b863-e45b-460b-9e49-1aae53077cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:55.934829 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.934814    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6417b863-e45b-460b-9e49-1aae53077cd5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9\" (UID: \"6417b863-e45b-460b-9e49-1aae53077cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:55.940791 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:55.940772    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpltq\" (UniqueName: \"kubernetes.io/projected/6417b863-e45b-460b-9e49-1aae53077cd5-kube-api-access-dpltq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9\" (UID: \"6417b863-e45b-460b-9e49-1aae53077cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:56.038621 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:56.038561    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:43:56.175029 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:56.175001    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"]
Aug 05 17:43:56.176663 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:43:56.176628    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6417b863_e45b_460b_9e49_1aae53077cd5.slice/crio-ac0dc8d02d2aa56c435045262605985a2cbf283ca58c5215e3f2e8e581cba041 WatchSource:0}: Error finding container ac0dc8d02d2aa56c435045262605985a2cbf283ca58c5215e3f2e8e581cba041: Status 404 returned error can't find the container with id ac0dc8d02d2aa56c435045262605985a2cbf283ca58c5215e3f2e8e581cba041
Aug 05 17:43:56.178001 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:56.177982    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:43:56.283285 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:56.283255    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9" event={"ID":"6417b863-e45b-460b-9e49-1aae53077cd5","Type":"ContainerStarted","Data":"ac0dc8d02d2aa56c435045262605985a2cbf283ca58c5215e3f2e8e581cba041"}
Aug 05 17:43:58.293597 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:58.293560    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9" event={"ID":"6417b863-e45b-460b-9e49-1aae53077cd5","Type":"ContainerStarted","Data":"44b08b420fc9a0b9e983d22584b2ec0d39fd7504b5f656e083209ee09c4d3f85"}
Aug 05 17:43:58.314081 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:43:58.314040    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9" podStartSLOduration=2.2521279 podStartE2EDuration="3.314025571s" totalImagesPullingTime="1.061897671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:43:58.311983656 +0000 UTC m=+645.664658250" watchObservedRunningTime="2026-08-05 17:43:58.314025571 +0000 UTC m=+645.666700164"
Aug 05 17:44:01.946457 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:01.946420    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:02.043461 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:02.043434    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:06.039289 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:06.039247    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:44:06.039717 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:06.039314    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:44:06.051810 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:06.051780    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:44:06.343524 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:06.343503    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9"
Aug 05 17:44:08.875293 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:08.875261    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:16.007950 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.007919    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"]
Aug 05 17:44:16.011794 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.011774    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.014348 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.014329    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Aug 05 17:44:16.022883 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.022861    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"]
Aug 05 17:44:16.050238 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.050207    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:16.109238 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.109212    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e83ba2d0-0799-4dd8-9f0e-72a321adf79e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6\" (UID: \"e83ba2d0-0799-4dd8-9f0e-72a321adf79e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.109367 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.109270    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e83ba2d0-0799-4dd8-9f0e-72a321adf79e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6\" (UID: \"e83ba2d0-0799-4dd8-9f0e-72a321adf79e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.109367 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.109298    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e83ba2d0-0799-4dd8-9f0e-72a321adf79e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6\" (UID: \"e83ba2d0-0799-4dd8-9f0e-72a321adf79e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.109491 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.109399    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e83ba2d0-0799-4dd8-9f0e-72a321adf79e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6\" (UID: \"e83ba2d0-0799-4dd8-9f0e-72a321adf79e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.109491 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.109458    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e83ba2d0-0799-4dd8-9f0e-72a321adf79e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6\" (UID: \"e83ba2d0-0799-4dd8-9f0e-72a321adf79e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.109589 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.109500    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzp8t\" (UniqueName: \"kubernetes.io/projected/e83ba2d0-0799-4dd8-9f0e-72a321adf79e-kube-api-access-nzp8t\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6\" (UID: \"e83ba2d0-0799-4dd8-9f0e-72a321adf79e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.210721 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.210695    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e83ba2d0-0799-4dd8-9f0e-72a321adf79e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6\" (UID: \"e83ba2d0-0799-4dd8-9f0e-72a321adf79e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.210821 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.210731    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzp8t\" (UniqueName: \"kubernetes.io/projected/e83ba2d0-0799-4dd8-9f0e-72a321adf79e-kube-api-access-nzp8t\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6\" (UID: \"e83ba2d0-0799-4dd8-9f0e-72a321adf79e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.210821 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.210770    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e83ba2d0-0799-4dd8-9f0e-72a321adf79e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6\" (UID: \"e83ba2d0-0799-4dd8-9f0e-72a321adf79e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.210924 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.210832    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e83ba2d0-0799-4dd8-9f0e-72a321adf79e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6\" (UID: \"e83ba2d0-0799-4dd8-9f0e-72a321adf79e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.210924 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.210858    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e83ba2d0-0799-4dd8-9f0e-72a321adf79e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6\" (UID: \"e83ba2d0-0799-4dd8-9f0e-72a321adf79e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.210924 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.210910    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e83ba2d0-0799-4dd8-9f0e-72a321adf79e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6\" (UID: \"e83ba2d0-0799-4dd8-9f0e-72a321adf79e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.211275 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.211248    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e83ba2d0-0799-4dd8-9f0e-72a321adf79e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6\" (UID: \"e83ba2d0-0799-4dd8-9f0e-72a321adf79e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.211375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.211299    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e83ba2d0-0799-4dd8-9f0e-72a321adf79e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6\" (UID: \"e83ba2d0-0799-4dd8-9f0e-72a321adf79e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.211375 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.211361    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e83ba2d0-0799-4dd8-9f0e-72a321adf79e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6\" (UID: \"e83ba2d0-0799-4dd8-9f0e-72a321adf79e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.213221 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.213201    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e83ba2d0-0799-4dd8-9f0e-72a321adf79e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6\" (UID: \"e83ba2d0-0799-4dd8-9f0e-72a321adf79e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.213621 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.213600    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e83ba2d0-0799-4dd8-9f0e-72a321adf79e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6\" (UID: \"e83ba2d0-0799-4dd8-9f0e-72a321adf79e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.219077 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.219048    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzp8t\" (UniqueName: \"kubernetes.io/projected/e83ba2d0-0799-4dd8-9f0e-72a321adf79e-kube-api-access-nzp8t\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6\" (UID: \"e83ba2d0-0799-4dd8-9f0e-72a321adf79e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.322100 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.322078    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:16.459036 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:16.459014    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"]
Aug 05 17:44:16.460934 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:44:16.460906    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode83ba2d0_0799_4dd8_9f0e_72a321adf79e.slice/crio-0afcfc89ad4ddbc39b8ff8f03f0363b4f2b0d1eebf61273e4a3d95ece14ed242 WatchSource:0}: Error finding container 0afcfc89ad4ddbc39b8ff8f03f0363b4f2b0d1eebf61273e4a3d95ece14ed242: Status 404 returned error can't find the container with id 0afcfc89ad4ddbc39b8ff8f03f0363b4f2b0d1eebf61273e4a3d95ece14ed242
Aug 05 17:44:17.390040 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:17.389998    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6" event={"ID":"e83ba2d0-0799-4dd8-9f0e-72a321adf79e","Type":"ContainerStarted","Data":"2b49fa67e1e770e16e8ae89a9324bdc0e5d271fe97e2bdb22fd05705e97cdd49"}
Aug 05 17:44:17.390040 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:17.390045    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6" event={"ID":"e83ba2d0-0799-4dd8-9f0e-72a321adf79e","Type":"ContainerStarted","Data":"0afcfc89ad4ddbc39b8ff8f03f0363b4f2b0d1eebf61273e4a3d95ece14ed242"}
Aug 05 17:44:17.411000 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:17.410945    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6" podStartSLOduration=2.191437354 podStartE2EDuration="2.410926222s" totalImagesPullingTime="219.488868ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:44:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:44:17.408453938 +0000 UTC m=+664.761128543" watchObservedRunningTime="2026-08-05 17:44:17.410926222 +0000 UTC m=+664.763600817"
Aug 05 17:44:26.323017 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:26.322981    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:26.323422 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:26.323040    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:26.336232 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:26.336207    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:26.450432 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:26.450402    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6"
Aug 05 17:44:28.275224 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.275174    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7646bfbd48-mhvks"]
Aug 05 17:44:28.281610 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.281586    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.292883 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.292858    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7646bfbd48-mhvks"]
Aug 05 17:44:28.337625 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.337599    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:28.398825 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.398796    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:28.420301 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.420274    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/737049a5-b68e-4c3e-ab9b-87aa444304c8-service-ca\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.420426 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.420303    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/737049a5-b68e-4c3e-ab9b-87aa444304c8-trusted-ca-bundle\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.420492 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.420416    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/737049a5-b68e-4c3e-ab9b-87aa444304c8-oauth-serving-cert\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.420492 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.420459    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/737049a5-b68e-4c3e-ab9b-87aa444304c8-console-oauth-config\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.420602 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.420572    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7mf6\" (UniqueName: \"kubernetes.io/projected/737049a5-b68e-4c3e-ab9b-87aa444304c8-kube-api-access-z7mf6\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.420656 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.420613    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/737049a5-b68e-4c3e-ab9b-87aa444304c8-console-serving-cert\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.420702 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.420680    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/737049a5-b68e-4c3e-ab9b-87aa444304c8-console-config\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.464306 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.464278    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:28.521782 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.521740    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7mf6\" (UniqueName: \"kubernetes.io/projected/737049a5-b68e-4c3e-ab9b-87aa444304c8-kube-api-access-z7mf6\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.521782 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.521788    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/737049a5-b68e-4c3e-ab9b-87aa444304c8-console-serving-cert\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.522003 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.521814    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/737049a5-b68e-4c3e-ab9b-87aa444304c8-console-config\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.522003 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.521841    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/737049a5-b68e-4c3e-ab9b-87aa444304c8-service-ca\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.522003 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.521861    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/737049a5-b68e-4c3e-ab9b-87aa444304c8-trusted-ca-bundle\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.522003 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.521923    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/737049a5-b68e-4c3e-ab9b-87aa444304c8-oauth-serving-cert\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.522003 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.521968    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/737049a5-b68e-4c3e-ab9b-87aa444304c8-console-oauth-config\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.522890 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.522859    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/737049a5-b68e-4c3e-ab9b-87aa444304c8-service-ca\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.523030 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.522896    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/737049a5-b68e-4c3e-ab9b-87aa444304c8-console-config\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.523030 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.522859    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/737049a5-b68e-4c3e-ab9b-87aa444304c8-trusted-ca-bundle\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.523030 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.522970    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/737049a5-b68e-4c3e-ab9b-87aa444304c8-oauth-serving-cert\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.524612 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.524592    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/737049a5-b68e-4c3e-ab9b-87aa444304c8-console-serving-cert\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.524700 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.524609    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/737049a5-b68e-4c3e-ab9b-87aa444304c8-console-oauth-config\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.530380 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.530322    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7mf6\" (UniqueName: \"kubernetes.io/projected/737049a5-b68e-4c3e-ab9b-87aa444304c8-kube-api-access-z7mf6\") pod \"console-7646bfbd48-mhvks\" (UID: \"737049a5-b68e-4c3e-ab9b-87aa444304c8\") " pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.566108 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.566070    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:28.592230 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.592157    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:28.742275 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:28.742133    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7646bfbd48-mhvks"]
Aug 05 17:44:28.744704 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:44:28.744679    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod737049a5_b68e_4c3e_ab9b_87aa444304c8.slice/crio-1b66dbfe9074bfe13743b7786adea0d477a68a4c6ddcf41d85a8e147e869db26 WatchSource:0}: Error finding container 1b66dbfe9074bfe13743b7786adea0d477a68a4c6ddcf41d85a8e147e869db26: Status 404 returned error can't find the container with id 1b66dbfe9074bfe13743b7786adea0d477a68a4c6ddcf41d85a8e147e869db26
Aug 05 17:44:29.061392 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:29.061306    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:29.131871 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:29.131840    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:29.456193 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:29.456156    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7646bfbd48-mhvks" event={"ID":"737049a5-b68e-4c3e-ab9b-87aa444304c8","Type":"ContainerStarted","Data":"c2d24bbe4d890d1795112b458212d683501df02ff404d5ce7d29fa0858d2c7ce"}
Aug 05 17:44:29.456679 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:29.456224    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7646bfbd48-mhvks" event={"ID":"737049a5-b68e-4c3e-ab9b-87aa444304c8","Type":"ContainerStarted","Data":"1b66dbfe9074bfe13743b7786adea0d477a68a4c6ddcf41d85a8e147e869db26"}
Aug 05 17:44:29.485151 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:29.485104    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7646bfbd48-mhvks" podStartSLOduration=1.485088101 podStartE2EDuration="1.485088101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:44:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:44:29.481703622 +0000 UTC m=+676.834378215" watchObservedRunningTime="2026-08-05 17:44:29.485088101 +0000 UTC m=+676.837762694"
Aug 05 17:44:38.593063 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:38.593026    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:38.593536 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:38.593309    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:38.597998 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:38.597975    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:39.509428 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:39.509394    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7646bfbd48-mhvks"
Aug 05 17:44:39.557811 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:39.557784    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-577d9b9fdf-dmcx2"]
Aug 05 17:44:39.673316 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:39.673253    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:39.773681 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:39.773601    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:39.873846 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:39.873820    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:39.973872 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:44:39.973843    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:45:04.582153 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.582079    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-577d9b9fdf-dmcx2" podUID="3859ed47-52a8-408b-8b10-c9771e05d349" containerName="console" containerID="cri-o://883358c92c6e6608a3a1dd215a2f87c8681e2d8c80cf5b3028b48f468ba03123" gracePeriod=15
Aug 05 17:45:04.824454 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.824430    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:45:04.946114 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.946055    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-oauth-serving-cert\") pod \"3859ed47-52a8-408b-8b10-c9771e05d349\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") "
Aug 05 17:45:04.946114 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.946083    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-console-config\") pod \"3859ed47-52a8-408b-8b10-c9771e05d349\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") "
Aug 05 17:45:04.946114 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.946108    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3859ed47-52a8-408b-8b10-c9771e05d349-console-oauth-config\") pod \"3859ed47-52a8-408b-8b10-c9771e05d349\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") "
Aug 05 17:45:04.946306 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.946140    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-trusted-ca-bundle\") pod \"3859ed47-52a8-408b-8b10-c9771e05d349\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") "
Aug 05 17:45:04.946306 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.946205    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-service-ca\") pod \"3859ed47-52a8-408b-8b10-c9771e05d349\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") "
Aug 05 17:45:04.946306 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.946282    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3859ed47-52a8-408b-8b10-c9771e05d349-console-serving-cert\") pod \"3859ed47-52a8-408b-8b10-c9771e05d349\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") "
Aug 05 17:45:04.946454 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.946316    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qknrl\" (UniqueName: \"kubernetes.io/projected/3859ed47-52a8-408b-8b10-c9771e05d349-kube-api-access-qknrl\") pod \"3859ed47-52a8-408b-8b10-c9771e05d349\" (UID: \"3859ed47-52a8-408b-8b10-c9771e05d349\") "
Aug 05 17:45:04.946618 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.946587    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3859ed47-52a8-408b-8b10-c9771e05d349" (UID: "3859ed47-52a8-408b-8b10-c9771e05d349"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:45:04.946618 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.946517    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-console-config" (OuterVolumeSpecName: "console-config") pod "3859ed47-52a8-408b-8b10-c9771e05d349" (UID: "3859ed47-52a8-408b-8b10-c9771e05d349"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:45:04.946778 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.946631    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3859ed47-52a8-408b-8b10-c9771e05d349" (UID: "3859ed47-52a8-408b-8b10-c9771e05d349"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:45:04.946778 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.946677    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-service-ca" (OuterVolumeSpecName: "service-ca") pod "3859ed47-52a8-408b-8b10-c9771e05d349" (UID: "3859ed47-52a8-408b-8b10-c9771e05d349"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:45:04.946887 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.946851    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-trusted-ca-bundle\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:45:04.946887 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.946864    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-service-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:45:04.946887 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.946873    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-oauth-serving-cert\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:45:04.946887 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.946881    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3859ed47-52a8-408b-8b10-c9771e05d349-console-config\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:45:04.948521 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.948493    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3859ed47-52a8-408b-8b10-c9771e05d349-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3859ed47-52a8-408b-8b10-c9771e05d349" (UID: "3859ed47-52a8-408b-8b10-c9771e05d349"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:45:04.948611 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.948530    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3859ed47-52a8-408b-8b10-c9771e05d349-kube-api-access-qknrl" (OuterVolumeSpecName: "kube-api-access-qknrl") pod "3859ed47-52a8-408b-8b10-c9771e05d349" (UID: "3859ed47-52a8-408b-8b10-c9771e05d349"). InnerVolumeSpecName "kube-api-access-qknrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:45:04.948611 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:04.948574    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3859ed47-52a8-408b-8b10-c9771e05d349-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3859ed47-52a8-408b-8b10-c9771e05d349" (UID: "3859ed47-52a8-408b-8b10-c9771e05d349"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:45:05.047772 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:05.047748    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3859ed47-52a8-408b-8b10-c9771e05d349-console-serving-cert\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:45:05.047772 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:05.047770    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qknrl\" (UniqueName: \"kubernetes.io/projected/3859ed47-52a8-408b-8b10-c9771e05d349-kube-api-access-qknrl\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:45:05.047920 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:05.047779    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3859ed47-52a8-408b-8b10-c9771e05d349-console-oauth-config\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:45:05.637657 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:05.637622    2569 generic.go:356] "Generic (PLEG): container finished" podID="3859ed47-52a8-408b-8b10-c9771e05d349" containerID="883358c92c6e6608a3a1dd215a2f87c8681e2d8c80cf5b3028b48f468ba03123" exitCode=0
Aug 05 17:45:05.638009 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:05.637669    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-577d9b9fdf-dmcx2" event={"ID":"3859ed47-52a8-408b-8b10-c9771e05d349","Type":"ContainerDied","Data":"883358c92c6e6608a3a1dd215a2f87c8681e2d8c80cf5b3028b48f468ba03123"}
Aug 05 17:45:05.638009 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:05.637677    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-577d9b9fdf-dmcx2"
Aug 05 17:45:05.638009 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:05.637740    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-577d9b9fdf-dmcx2" event={"ID":"3859ed47-52a8-408b-8b10-c9771e05d349","Type":"ContainerDied","Data":"db7c1c20baee8a620ee892eacc3a9da7a4526b21fd0737660d32626cfde37052"}
Aug 05 17:45:05.638009 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:05.637761    2569 scope.go:117] "RemoveContainer" containerID="883358c92c6e6608a3a1dd215a2f87c8681e2d8c80cf5b3028b48f468ba03123"
Aug 05 17:45:05.646670 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:05.646650    2569 scope.go:117] "RemoveContainer" containerID="883358c92c6e6608a3a1dd215a2f87c8681e2d8c80cf5b3028b48f468ba03123"
Aug 05 17:45:05.646931 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:45:05.646914    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"883358c92c6e6608a3a1dd215a2f87c8681e2d8c80cf5b3028b48f468ba03123\": container with ID starting with 883358c92c6e6608a3a1dd215a2f87c8681e2d8c80cf5b3028b48f468ba03123 not found: ID does not exist" containerID="883358c92c6e6608a3a1dd215a2f87c8681e2d8c80cf5b3028b48f468ba03123"
Aug 05 17:45:05.646974 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:05.646941    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"883358c92c6e6608a3a1dd215a2f87c8681e2d8c80cf5b3028b48f468ba03123"} err="failed to get container status \"883358c92c6e6608a3a1dd215a2f87c8681e2d8c80cf5b3028b48f468ba03123\": rpc error: code = NotFound desc = could not find container \"883358c92c6e6608a3a1dd215a2f87c8681e2d8c80cf5b3028b48f468ba03123\": container with ID starting with 883358c92c6e6608a3a1dd215a2f87c8681e2d8c80cf5b3028b48f468ba03123 not found: ID does not exist"
Aug 05 17:45:05.654739 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:05.654719    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-577d9b9fdf-dmcx2"]
Aug 05 17:45:05.658372 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:05.658344    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-577d9b9fdf-dmcx2"]
Aug 05 17:45:07.150675 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:07.150636    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3859ed47-52a8-408b-8b10-c9771e05d349" path="/var/lib/kubelet/pods/3859ed47-52a8-408b-8b10-c9771e05d349/volumes"
Aug 05 17:45:21.687705 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:21.687675    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5c844f45b7-mwq9n"]
Aug 05 17:45:21.688082 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:21.688070    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3859ed47-52a8-408b-8b10-c9771e05d349" containerName="console"
Aug 05 17:45:21.688126 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:21.688085    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3859ed47-52a8-408b-8b10-c9771e05d349" containerName="console"
Aug 05 17:45:21.688205 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:21.688173    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3859ed47-52a8-408b-8b10-c9771e05d349" containerName="console"
Aug 05 17:45:21.692690 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:21.692673    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c844f45b7-mwq9n"
Aug 05 17:45:21.695820 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:21.695802    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-99fvc\""
Aug 05 17:45:21.697516 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:21.697489    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Aug 05 17:45:21.699508 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:21.699484    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c844f45b7-mwq9n"]
Aug 05 17:45:21.783430 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:21.783387    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b8d3e1c1-116a-43a7-b8f2-ba006085541d-tls-cert\") pod \"authorino-5c844f45b7-mwq9n\" (UID: \"b8d3e1c1-116a-43a7-b8f2-ba006085541d\") " pod="kuadrant-system/authorino-5c844f45b7-mwq9n"
Aug 05 17:45:21.783577 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:21.783460    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlk6k\" (UniqueName: \"kubernetes.io/projected/b8d3e1c1-116a-43a7-b8f2-ba006085541d-kube-api-access-nlk6k\") pod \"authorino-5c844f45b7-mwq9n\" (UID: \"b8d3e1c1-116a-43a7-b8f2-ba006085541d\") " pod="kuadrant-system/authorino-5c844f45b7-mwq9n"
Aug 05 17:45:21.884228 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:21.884155    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b8d3e1c1-116a-43a7-b8f2-ba006085541d-tls-cert\") pod \"authorino-5c844f45b7-mwq9n\" (UID: \"b8d3e1c1-116a-43a7-b8f2-ba006085541d\") " pod="kuadrant-system/authorino-5c844f45b7-mwq9n"
Aug 05 17:45:21.884405 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:21.884277    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlk6k\" (UniqueName: \"kubernetes.io/projected/b8d3e1c1-116a-43a7-b8f2-ba006085541d-kube-api-access-nlk6k\") pod \"authorino-5c844f45b7-mwq9n\" (UID: \"b8d3e1c1-116a-43a7-b8f2-ba006085541d\") " pod="kuadrant-system/authorino-5c844f45b7-mwq9n"
Aug 05 17:45:21.886813 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:21.886786    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b8d3e1c1-116a-43a7-b8f2-ba006085541d-tls-cert\") pod \"authorino-5c844f45b7-mwq9n\" (UID: \"b8d3e1c1-116a-43a7-b8f2-ba006085541d\") " pod="kuadrant-system/authorino-5c844f45b7-mwq9n"
Aug 05 17:45:21.893165 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:21.893143    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlk6k\" (UniqueName: \"kubernetes.io/projected/b8d3e1c1-116a-43a7-b8f2-ba006085541d-kube-api-access-nlk6k\") pod \"authorino-5c844f45b7-mwq9n\" (UID: \"b8d3e1c1-116a-43a7-b8f2-ba006085541d\") " pod="kuadrant-system/authorino-5c844f45b7-mwq9n"
Aug 05 17:45:22.004080 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:22.003995    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c844f45b7-mwq9n"
Aug 05 17:45:22.141893 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:22.141867    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c844f45b7-mwq9n"]
Aug 05 17:45:22.144388 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:45:22.144361    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8d3e1c1_116a_43a7_b8f2_ba006085541d.slice/crio-bf1504712f8049fcd31d83f2fbe3663c79459ff1167ff56605306d3393ba1bed WatchSource:0}: Error finding container bf1504712f8049fcd31d83f2fbe3663c79459ff1167ff56605306d3393ba1bed: Status 404 returned error can't find the container with id bf1504712f8049fcd31d83f2fbe3663c79459ff1167ff56605306d3393ba1bed
Aug 05 17:45:22.729679 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:22.729645    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c844f45b7-mwq9n" event={"ID":"b8d3e1c1-116a-43a7-b8f2-ba006085541d","Type":"ContainerStarted","Data":"bf1504712f8049fcd31d83f2fbe3663c79459ff1167ff56605306d3393ba1bed"}
Aug 05 17:45:30.768778 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:30.768745    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c844f45b7-mwq9n" event={"ID":"b8d3e1c1-116a-43a7-b8f2-ba006085541d","Type":"ContainerStarted","Data":"c75d3f0bf082c06957c2935e81a1fb32224b6c8c717f98fcf76a822b569c17bd"}
Aug 05 17:45:30.827383 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:30.827315    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5c844f45b7-mwq9n" podStartSLOduration=1.754962476 podStartE2EDuration="9.827291736s" totalImagesPullingTime="8.07232926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:45:30.826396359 +0000 UTC m=+738.179070973" watchObservedRunningTime="2026-08-05 17:45:30.827291736 +0000 UTC m=+738.179966328"
Aug 05 17:45:31.435584 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:31.435553    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:45:31.536160 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:31.536122    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:45:32.510917 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:32.510886    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:45:32.609673 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:45:32.609641    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:46:19.625867 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:19.625830    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67657dfbc-jc99s"]
Aug 05 17:46:19.626348 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:19.626066    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67657dfbc-jc99s" podUID="de9f555f-7afc-4a8a-be1e-1ff50e0efd5e" containerName="manager" containerID="cri-o://31e59b5395ee3a8f56710a900304073ae96e9945c575c39de57297e6148fa450" gracePeriod=30
Aug 05 17:46:19.878385 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:19.878327    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67657dfbc-jc99s"
Aug 05 17:46:19.922443 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:19.922411    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de9f555f-7afc-4a8a-be1e-1ff50e0efd5e-webhook-cert\") pod \"de9f555f-7afc-4a8a-be1e-1ff50e0efd5e\" (UID: \"de9f555f-7afc-4a8a-be1e-1ff50e0efd5e\") "
Aug 05 17:46:19.922618 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:19.922463    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v7s8m\" (UniqueName: \"kubernetes.io/projected/de9f555f-7afc-4a8a-be1e-1ff50e0efd5e-kube-api-access-v7s8m\") pod \"de9f555f-7afc-4a8a-be1e-1ff50e0efd5e\" (UID: \"de9f555f-7afc-4a8a-be1e-1ff50e0efd5e\") "
Aug 05 17:46:19.924722 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:19.924691    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de9f555f-7afc-4a8a-be1e-1ff50e0efd5e-kube-api-access-v7s8m" (OuterVolumeSpecName: "kube-api-access-v7s8m") pod "de9f555f-7afc-4a8a-be1e-1ff50e0efd5e" (UID: "de9f555f-7afc-4a8a-be1e-1ff50e0efd5e"). InnerVolumeSpecName "kube-api-access-v7s8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:46:19.924846 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:19.924806    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de9f555f-7afc-4a8a-be1e-1ff50e0efd5e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "de9f555f-7afc-4a8a-be1e-1ff50e0efd5e" (UID: "de9f555f-7afc-4a8a-be1e-1ff50e0efd5e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:46:20.023589 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:20.023560    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7s8m\" (UniqueName: \"kubernetes.io/projected/de9f555f-7afc-4a8a-be1e-1ff50e0efd5e-kube-api-access-v7s8m\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:46:20.023589 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:20.023583    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de9f555f-7afc-4a8a-be1e-1ff50e0efd5e-webhook-cert\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:46:20.028790 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:20.028761    2569 generic.go:356] "Generic (PLEG): container finished" podID="de9f555f-7afc-4a8a-be1e-1ff50e0efd5e" containerID="31e59b5395ee3a8f56710a900304073ae96e9945c575c39de57297e6148fa450" exitCode=0
Aug 05 17:46:20.028937 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:20.028800    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67657dfbc-jc99s" event={"ID":"de9f555f-7afc-4a8a-be1e-1ff50e0efd5e","Type":"ContainerDied","Data":"31e59b5395ee3a8f56710a900304073ae96e9945c575c39de57297e6148fa450"}
Aug 05 17:46:20.028937 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:20.028860    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67657dfbc-jc99s" event={"ID":"de9f555f-7afc-4a8a-be1e-1ff50e0efd5e","Type":"ContainerDied","Data":"9ecebafd5c241d0d12b7156ee9d20ce3bdebcbac5fb86cfd41bcc7db428bb7c3"}
Aug 05 17:46:20.028937 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:20.028814    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67657dfbc-jc99s"
Aug 05 17:46:20.028937 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:20.028895    2569 scope.go:117] "RemoveContainer" containerID="31e59b5395ee3a8f56710a900304073ae96e9945c575c39de57297e6148fa450"
Aug 05 17:46:20.039304 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:20.039261    2569 scope.go:117] "RemoveContainer" containerID="31e59b5395ee3a8f56710a900304073ae96e9945c575c39de57297e6148fa450"
Aug 05 17:46:20.039532 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:46:20.039515    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31e59b5395ee3a8f56710a900304073ae96e9945c575c39de57297e6148fa450\": container with ID starting with 31e59b5395ee3a8f56710a900304073ae96e9945c575c39de57297e6148fa450 not found: ID does not exist" containerID="31e59b5395ee3a8f56710a900304073ae96e9945c575c39de57297e6148fa450"
Aug 05 17:46:20.039584 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:20.039540    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31e59b5395ee3a8f56710a900304073ae96e9945c575c39de57297e6148fa450"} err="failed to get container status \"31e59b5395ee3a8f56710a900304073ae96e9945c575c39de57297e6148fa450\": rpc error: code = NotFound desc = could not find container \"31e59b5395ee3a8f56710a900304073ae96e9945c575c39de57297e6148fa450\": container with ID starting with 31e59b5395ee3a8f56710a900304073ae96e9945c575c39de57297e6148fa450 not found: ID does not exist"
Aug 05 17:46:20.049694 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:20.049674    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67657dfbc-jc99s"]
Aug 05 17:46:20.052811 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:20.052792    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67657dfbc-jc99s"]
Aug 05 17:46:21.151355 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:21.151319    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de9f555f-7afc-4a8a-be1e-1ff50e0efd5e" path="/var/lib/kubelet/pods/de9f555f-7afc-4a8a-be1e-1ff50e0efd5e/volumes"
Aug 05 17:46:58.452013 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:58.451974    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:46:58.554282 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:46:58.554256    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:47:20.842131 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:20.842092    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67657dfbc-n2hr5"]
Aug 05 17:47:20.842599 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:20.842558    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de9f555f-7afc-4a8a-be1e-1ff50e0efd5e" containerName="manager"
Aug 05 17:47:20.842599 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:20.842572    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="de9f555f-7afc-4a8a-be1e-1ff50e0efd5e" containerName="manager"
Aug 05 17:47:20.842669 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:20.842653    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="de9f555f-7afc-4a8a-be1e-1ff50e0efd5e" containerName="manager"
Aug 05 17:47:20.846127 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:20.846112    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67657dfbc-n2hr5"
Aug 05 17:47:20.849871 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:20.849849    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 05 17:47:20.850012 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:20.849851    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ssqzs\""
Aug 05 17:47:20.850012 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:20.849903    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 05 17:47:20.855140 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:20.855121    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67657dfbc-n2hr5"]
Aug 05 17:47:20.979776 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:20.979743    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/83319764-e291-46e8-8d15-e1631c089915-webhook-cert\") pod \"maas-controller-67657dfbc-n2hr5\" (UID: \"83319764-e291-46e8-8d15-e1631c089915\") " pod="opendatahub/maas-controller-67657dfbc-n2hr5"
Aug 05 17:47:20.979776 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:20.979782    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rszww\" (UniqueName: \"kubernetes.io/projected/83319764-e291-46e8-8d15-e1631c089915-kube-api-access-rszww\") pod \"maas-controller-67657dfbc-n2hr5\" (UID: \"83319764-e291-46e8-8d15-e1631c089915\") " pod="opendatahub/maas-controller-67657dfbc-n2hr5"
Aug 05 17:47:21.080559 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:21.080526    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/83319764-e291-46e8-8d15-e1631c089915-webhook-cert\") pod \"maas-controller-67657dfbc-n2hr5\" (UID: \"83319764-e291-46e8-8d15-e1631c089915\") " pod="opendatahub/maas-controller-67657dfbc-n2hr5"
Aug 05 17:47:21.080743 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:21.080567    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rszww\" (UniqueName: \"kubernetes.io/projected/83319764-e291-46e8-8d15-e1631c089915-kube-api-access-rszww\") pod \"maas-controller-67657dfbc-n2hr5\" (UID: \"83319764-e291-46e8-8d15-e1631c089915\") " pod="opendatahub/maas-controller-67657dfbc-n2hr5"
Aug 05 17:47:21.083138 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:21.083115    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/83319764-e291-46e8-8d15-e1631c089915-webhook-cert\") pod \"maas-controller-67657dfbc-n2hr5\" (UID: \"83319764-e291-46e8-8d15-e1631c089915\") " pod="opendatahub/maas-controller-67657dfbc-n2hr5"
Aug 05 17:47:21.090598 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:21.090562    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rszww\" (UniqueName: \"kubernetes.io/projected/83319764-e291-46e8-8d15-e1631c089915-kube-api-access-rszww\") pod \"maas-controller-67657dfbc-n2hr5\" (UID: \"83319764-e291-46e8-8d15-e1631c089915\") " pod="opendatahub/maas-controller-67657dfbc-n2hr5"
Aug 05 17:47:21.157173 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:21.157111    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67657dfbc-n2hr5"
Aug 05 17:47:21.295912 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:21.295887    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67657dfbc-n2hr5"]
Aug 05 17:47:21.298362 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:47:21.298329    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83319764_e291_46e8_8d15_e1631c089915.slice/crio-b0df7ce5105cee5e7a94772e2105fbc7f14647aa1a3d5f917da3fc9098fee16c WatchSource:0}: Error finding container b0df7ce5105cee5e7a94772e2105fbc7f14647aa1a3d5f917da3fc9098fee16c: Status 404 returned error can't find the container with id b0df7ce5105cee5e7a94772e2105fbc7f14647aa1a3d5f917da3fc9098fee16c
Aug 05 17:47:21.343297 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:21.343271    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67657dfbc-n2hr5" event={"ID":"83319764-e291-46e8-8d15-e1631c089915","Type":"ContainerStarted","Data":"b0df7ce5105cee5e7a94772e2105fbc7f14647aa1a3d5f917da3fc9098fee16c"}
Aug 05 17:47:22.352440 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:22.352403    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67657dfbc-n2hr5" event={"ID":"83319764-e291-46e8-8d15-e1631c089915","Type":"ContainerStarted","Data":"fb8731c784c87d301b60964b5154c9af71bc867566cd95a56e83570fb4591c30"}
Aug 05 17:47:22.352830 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:22.352550    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67657dfbc-n2hr5"
Aug 05 17:47:22.379710 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:22.379667    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67657dfbc-n2hr5" podStartSLOduration=1.909470596 podStartE2EDuration="2.379654197s" totalImagesPullingTime="470.183601ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:47:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:47:22.378473849 +0000 UTC m=+849.731148440" watchObservedRunningTime="2026-08-05 17:47:22.379654197 +0000 UTC m=+849.732328791"
Aug 05 17:47:33.359139 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:33.359112    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67657dfbc-n2hr5"
Aug 05 17:47:37.881373 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:37.881338    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:47:37.967838 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:37.967812    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67657dfbc-n2hr5"]
Aug 05 17:47:37.968083 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:37.968043    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67657dfbc-n2hr5" podUID="83319764-e291-46e8-8d15-e1631c089915" containerName="manager" containerID="cri-o://fb8731c784c87d301b60964b5154c9af71bc867566cd95a56e83570fb4591c30" gracePeriod=30
Aug 05 17:47:38.222353 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:38.222330    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67657dfbc-n2hr5"
Aug 05 17:47:38.331858 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:38.331831    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/83319764-e291-46e8-8d15-e1631c089915-webhook-cert\") pod \"83319764-e291-46e8-8d15-e1631c089915\" (UID: \"83319764-e291-46e8-8d15-e1631c089915\") "
Aug 05 17:47:38.332024 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:38.331878    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rszww\" (UniqueName: \"kubernetes.io/projected/83319764-e291-46e8-8d15-e1631c089915-kube-api-access-rszww\") pod \"83319764-e291-46e8-8d15-e1631c089915\" (UID: \"83319764-e291-46e8-8d15-e1631c089915\") "
Aug 05 17:47:38.333905 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:38.333873    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83319764-e291-46e8-8d15-e1631c089915-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "83319764-e291-46e8-8d15-e1631c089915" (UID: "83319764-e291-46e8-8d15-e1631c089915"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:47:38.334025 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:38.334007    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83319764-e291-46e8-8d15-e1631c089915-kube-api-access-rszww" (OuterVolumeSpecName: "kube-api-access-rszww") pod "83319764-e291-46e8-8d15-e1631c089915" (UID: "83319764-e291-46e8-8d15-e1631c089915"). InnerVolumeSpecName "kube-api-access-rszww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:47:38.432760 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:38.432669    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/83319764-e291-46e8-8d15-e1631c089915-webhook-cert\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:47:38.432760 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:38.432701    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rszww\" (UniqueName: \"kubernetes.io/projected/83319764-e291-46e8-8d15-e1631c089915-kube-api-access-rszww\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:47:38.433172 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:38.433150    2569 generic.go:356] "Generic (PLEG): container finished" podID="83319764-e291-46e8-8d15-e1631c089915" containerID="fb8731c784c87d301b60964b5154c9af71bc867566cd95a56e83570fb4591c30" exitCode=0
Aug 05 17:47:38.433253 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:38.433211    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67657dfbc-n2hr5" event={"ID":"83319764-e291-46e8-8d15-e1631c089915","Type":"ContainerDied","Data":"fb8731c784c87d301b60964b5154c9af71bc867566cd95a56e83570fb4591c30"}
Aug 05 17:47:38.433253 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:38.433244    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67657dfbc-n2hr5"
Aug 05 17:47:38.433331 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:38.433268    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67657dfbc-n2hr5" event={"ID":"83319764-e291-46e8-8d15-e1631c089915","Type":"ContainerDied","Data":"b0df7ce5105cee5e7a94772e2105fbc7f14647aa1a3d5f917da3fc9098fee16c"}
Aug 05 17:47:38.433331 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:38.433293    2569 scope.go:117] "RemoveContainer" containerID="fb8731c784c87d301b60964b5154c9af71bc867566cd95a56e83570fb4591c30"
Aug 05 17:47:38.442191 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:38.442160    2569 scope.go:117] "RemoveContainer" containerID="fb8731c784c87d301b60964b5154c9af71bc867566cd95a56e83570fb4591c30"
Aug 05 17:47:38.442432 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:47:38.442417    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb8731c784c87d301b60964b5154c9af71bc867566cd95a56e83570fb4591c30\": container with ID starting with fb8731c784c87d301b60964b5154c9af71bc867566cd95a56e83570fb4591c30 not found: ID does not exist" containerID="fb8731c784c87d301b60964b5154c9af71bc867566cd95a56e83570fb4591c30"
Aug 05 17:47:38.442489 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:38.442439    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb8731c784c87d301b60964b5154c9af71bc867566cd95a56e83570fb4591c30"} err="failed to get container status \"fb8731c784c87d301b60964b5154c9af71bc867566cd95a56e83570fb4591c30\": rpc error: code = NotFound desc = could not find container \"fb8731c784c87d301b60964b5154c9af71bc867566cd95a56e83570fb4591c30\": container with ID starting with fb8731c784c87d301b60964b5154c9af71bc867566cd95a56e83570fb4591c30 not found: ID does not exist"
Aug 05 17:47:38.456288 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:38.456267    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67657dfbc-n2hr5"]
Aug 05 17:47:38.460346 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:38.460326    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67657dfbc-n2hr5"]
Aug 05 17:47:39.114755 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:39.114720    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:47:39.150662 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:39.150634    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83319764-e291-46e8-8d15-e1631c089915" path="/var/lib/kubelet/pods/83319764-e291-46e8-8d15-e1631c089915/volumes"
Aug 05 17:47:39.240561 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:39.240534    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67657dfbc-g4pbp"]
Aug 05 17:47:39.240929 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:39.240917    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83319764-e291-46e8-8d15-e1631c089915" containerName="manager"
Aug 05 17:47:39.240974 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:39.240931    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="83319764-e291-46e8-8d15-e1631c089915" containerName="manager"
Aug 05 17:47:39.241015 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:39.241006    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="83319764-e291-46e8-8d15-e1631c089915" containerName="manager"
Aug 05 17:47:39.245555 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:39.245537    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67657dfbc-g4pbp"
Aug 05 17:47:39.248361 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:39.248344    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 05 17:47:39.248927 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:39.248909    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 05 17:47:39.249508 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:39.249495    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ssqzs\""
Aug 05 17:47:39.256932 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:39.256914    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67657dfbc-g4pbp"]
Aug 05 17:47:39.340273 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:39.340248    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99c2m\" (UniqueName: \"kubernetes.io/projected/2c3bca01-3915-4c42-8478-39f4b3a87950-kube-api-access-99c2m\") pod \"maas-controller-67657dfbc-g4pbp\" (UID: \"2c3bca01-3915-4c42-8478-39f4b3a87950\") " pod="opendatahub/maas-controller-67657dfbc-g4pbp"
Aug 05 17:47:39.340428 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:39.340286    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c3bca01-3915-4c42-8478-39f4b3a87950-webhook-cert\") pod \"maas-controller-67657dfbc-g4pbp\" (UID: \"2c3bca01-3915-4c42-8478-39f4b3a87950\") " pod="opendatahub/maas-controller-67657dfbc-g4pbp"
Aug 05 17:47:39.441459 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:39.441394    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c3bca01-3915-4c42-8478-39f4b3a87950-webhook-cert\") pod \"maas-controller-67657dfbc-g4pbp\" (UID: \"2c3bca01-3915-4c42-8478-39f4b3a87950\") " pod="opendatahub/maas-controller-67657dfbc-g4pbp"
Aug 05 17:47:39.441562 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:39.441472    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99c2m\" (UniqueName: \"kubernetes.io/projected/2c3bca01-3915-4c42-8478-39f4b3a87950-kube-api-access-99c2m\") pod \"maas-controller-67657dfbc-g4pbp\" (UID: \"2c3bca01-3915-4c42-8478-39f4b3a87950\") " pod="opendatahub/maas-controller-67657dfbc-g4pbp"
Aug 05 17:47:39.443850 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:39.443832    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c3bca01-3915-4c42-8478-39f4b3a87950-webhook-cert\") pod \"maas-controller-67657dfbc-g4pbp\" (UID: \"2c3bca01-3915-4c42-8478-39f4b3a87950\") " pod="opendatahub/maas-controller-67657dfbc-g4pbp"
Aug 05 17:47:39.450131 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:39.450109    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99c2m\" (UniqueName: \"kubernetes.io/projected/2c3bca01-3915-4c42-8478-39f4b3a87950-kube-api-access-99c2m\") pod \"maas-controller-67657dfbc-g4pbp\" (UID: \"2c3bca01-3915-4c42-8478-39f4b3a87950\") " pod="opendatahub/maas-controller-67657dfbc-g4pbp"
Aug 05 17:47:39.555871 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:39.555849    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67657dfbc-g4pbp"
Aug 05 17:47:39.686953 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:39.686925    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67657dfbc-g4pbp"]
Aug 05 17:47:39.689088 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:47:39.689048    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c3bca01_3915_4c42_8478_39f4b3a87950.slice/crio-8a2a7978e430ded959fb4b68adde2bc4489aecab0c5f9dee9f04653ae4fab337 WatchSource:0}: Error finding container 8a2a7978e430ded959fb4b68adde2bc4489aecab0c5f9dee9f04653ae4fab337: Status 404 returned error can't find the container with id 8a2a7978e430ded959fb4b68adde2bc4489aecab0c5f9dee9f04653ae4fab337
Aug 05 17:47:40.444573 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:40.444535    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67657dfbc-g4pbp" event={"ID":"2c3bca01-3915-4c42-8478-39f4b3a87950","Type":"ContainerStarted","Data":"8a2a7978e430ded959fb4b68adde2bc4489aecab0c5f9dee9f04653ae4fab337"}
Aug 05 17:47:41.450736 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:41.450701    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67657dfbc-g4pbp" event={"ID":"2c3bca01-3915-4c42-8478-39f4b3a87950","Type":"ContainerStarted","Data":"ea86ace086d3e25306af64978c262c6f2d9fd2466ee4f9530e6ac6fa4b50d593"}
Aug 05 17:47:41.451099 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:41.450783    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67657dfbc-g4pbp"
Aug 05 17:47:41.475407 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:41.475362    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67657dfbc-g4pbp" podStartSLOduration=1.4332151770000001 podStartE2EDuration="2.475349014s" totalImagesPullingTime="1.042133837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:47:41.472948847 +0000 UTC m=+868.825623460" watchObservedRunningTime="2026-08-05 17:47:41.475349014 +0000 UTC m=+868.828023608"
Aug 05 17:47:52.457909 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:52.457877    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67657dfbc-g4pbp"
Aug 05 17:47:59.132982 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:59.132947    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:47:59.195644 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:47:59.195616    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:49.545789 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:48:49.545753    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:51.128068 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:48:51.128008    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:56.088775 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:48:56.088747    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:56.195885 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:48:56.195849    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:56.291585 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:48:56.291549    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:56.430043 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:48:56.429974    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:49:02.776515 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:49:02.776485    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:49:02.879204 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:49:02.879148    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:49:26.425548 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:49:26.425512    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:49:26.486308 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:49:26.486285    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:50:47.714776 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:50:47.714740    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:50:48.914925 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:50:48.914893    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:50:53.800532 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:50:53.800505    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:20.776233 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:20.776132    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx"]
Aug 05 17:54:20.778683 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:20.776353    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx" podUID="e5d029d0-2470-4054-8ef9-4d3eb9aaf600" containerName="manager" containerID="cri-o://d682cad31201b572f3bf73c30ed7de01fd770cdb14f62331ce79ec5634b48dc3" gracePeriod=10
Aug 05 17:54:21.033513 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:21.033462    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx"
Aug 05 17:54:21.169430 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:21.169404    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwtf5\" (UniqueName: \"kubernetes.io/projected/e5d029d0-2470-4054-8ef9-4d3eb9aaf600-kube-api-access-hwtf5\") pod \"e5d029d0-2470-4054-8ef9-4d3eb9aaf600\" (UID: \"e5d029d0-2470-4054-8ef9-4d3eb9aaf600\") "
Aug 05 17:54:21.169591 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:21.169461    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5d029d0-2470-4054-8ef9-4d3eb9aaf600-extensions-socket-volume\") pod \"e5d029d0-2470-4054-8ef9-4d3eb9aaf600\" (UID: \"e5d029d0-2470-4054-8ef9-4d3eb9aaf600\") "
Aug 05 17:54:21.169866 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:21.169840    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5d029d0-2470-4054-8ef9-4d3eb9aaf600-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e5d029d0-2470-4054-8ef9-4d3eb9aaf600" (UID: "e5d029d0-2470-4054-8ef9-4d3eb9aaf600"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:54:21.171567 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:21.171543    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5d029d0-2470-4054-8ef9-4d3eb9aaf600-kube-api-access-hwtf5" (OuterVolumeSpecName: "kube-api-access-hwtf5") pod "e5d029d0-2470-4054-8ef9-4d3eb9aaf600" (UID: "e5d029d0-2470-4054-8ef9-4d3eb9aaf600"). InnerVolumeSpecName "kube-api-access-hwtf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:54:21.270661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:21.270634    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwtf5\" (UniqueName: \"kubernetes.io/projected/e5d029d0-2470-4054-8ef9-4d3eb9aaf600-kube-api-access-hwtf5\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:54:21.270661 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:21.270658    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5d029d0-2470-4054-8ef9-4d3eb9aaf600-extensions-socket-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:54:21.476540 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:21.476514    2569 generic.go:356] "Generic (PLEG): container finished" podID="e5d029d0-2470-4054-8ef9-4d3eb9aaf600" containerID="d682cad31201b572f3bf73c30ed7de01fd770cdb14f62331ce79ec5634b48dc3" exitCode=0
Aug 05 17:54:21.476671 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:21.476562    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx"
Aug 05 17:54:21.476671 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:21.476601    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx" event={"ID":"e5d029d0-2470-4054-8ef9-4d3eb9aaf600","Type":"ContainerDied","Data":"d682cad31201b572f3bf73c30ed7de01fd770cdb14f62331ce79ec5634b48dc3"}
Aug 05 17:54:21.476671 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:21.476655    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx" event={"ID":"e5d029d0-2470-4054-8ef9-4d3eb9aaf600","Type":"ContainerDied","Data":"4287b37449e4ec12951ad061028d544b1cab30d5d2f0fb5e01592fede9b88212"}
Aug 05 17:54:21.476856 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:21.476678    2569 scope.go:117] "RemoveContainer" containerID="d682cad31201b572f3bf73c30ed7de01fd770cdb14f62331ce79ec5634b48dc3"
Aug 05 17:54:21.486587 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:21.486570    2569 scope.go:117] "RemoveContainer" containerID="d682cad31201b572f3bf73c30ed7de01fd770cdb14f62331ce79ec5634b48dc3"
Aug 05 17:54:21.486823 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:54:21.486805    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d682cad31201b572f3bf73c30ed7de01fd770cdb14f62331ce79ec5634b48dc3\": container with ID starting with d682cad31201b572f3bf73c30ed7de01fd770cdb14f62331ce79ec5634b48dc3 not found: ID does not exist" containerID="d682cad31201b572f3bf73c30ed7de01fd770cdb14f62331ce79ec5634b48dc3"
Aug 05 17:54:21.486870 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:21.486831    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d682cad31201b572f3bf73c30ed7de01fd770cdb14f62331ce79ec5634b48dc3"} err="failed to get container status \"d682cad31201b572f3bf73c30ed7de01fd770cdb14f62331ce79ec5634b48dc3\": rpc error: code = NotFound desc = could not find container \"d682cad31201b572f3bf73c30ed7de01fd770cdb14f62331ce79ec5634b48dc3\": container with ID starting with d682cad31201b572f3bf73c30ed7de01fd770cdb14f62331ce79ec5634b48dc3 not found: ID does not exist"
Aug 05 17:54:21.506767 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:21.506747    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx"]
Aug 05 17:54:21.512946 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:21.512925    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2j6wx"]
Aug 05 17:54:23.150074 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:23.150045    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5d029d0-2470-4054-8ef9-4d3eb9aaf600" path="/var/lib/kubelet/pods/e5d029d0-2470-4054-8ef9-4d3eb9aaf600/volumes"
Aug 05 17:54:53.846100 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:54:53.846063    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:55:21.469497 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:21.469463    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x"]
Aug 05 17:55:21.469898 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:21.469884    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5d029d0-2470-4054-8ef9-4d3eb9aaf600" containerName="manager"
Aug 05 17:55:21.469945 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:21.469903    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5d029d0-2470-4054-8ef9-4d3eb9aaf600" containerName="manager"
Aug 05 17:55:21.470004 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:21.469995    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5d029d0-2470-4054-8ef9-4d3eb9aaf600" containerName="manager"
Aug 05 17:55:21.474161 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:21.474145    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x"
Aug 05 17:55:21.476991 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:21.476961    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hfkjp\""
Aug 05 17:55:21.484938 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:21.484913    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x"]
Aug 05 17:55:21.565244 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:21.565222    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/66eb3ae5-5efe-4070-a999-d399e0a875f0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-bkf7x\" (UID: \"66eb3ae5-5efe-4070-a999-d399e0a875f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x"
Aug 05 17:55:21.565353 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:21.565293    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfj7j\" (UniqueName: \"kubernetes.io/projected/66eb3ae5-5efe-4070-a999-d399e0a875f0-kube-api-access-mfj7j\") pod \"kuadrant-operator-controller-manager-68cf589589-bkf7x\" (UID: \"66eb3ae5-5efe-4070-a999-d399e0a875f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x"
Aug 05 17:55:21.666465 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:21.666440    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfj7j\" (UniqueName: \"kubernetes.io/projected/66eb3ae5-5efe-4070-a999-d399e0a875f0-kube-api-access-mfj7j\") pod \"kuadrant-operator-controller-manager-68cf589589-bkf7x\" (UID: \"66eb3ae5-5efe-4070-a999-d399e0a875f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x"
Aug 05 17:55:21.666554 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:21.666489    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/66eb3ae5-5efe-4070-a999-d399e0a875f0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-bkf7x\" (UID: \"66eb3ae5-5efe-4070-a999-d399e0a875f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x"
Aug 05 17:55:21.666802 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:21.666787    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/66eb3ae5-5efe-4070-a999-d399e0a875f0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-bkf7x\" (UID: \"66eb3ae5-5efe-4070-a999-d399e0a875f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x"
Aug 05 17:55:21.674995 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:21.674970    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfj7j\" (UniqueName: \"kubernetes.io/projected/66eb3ae5-5efe-4070-a999-d399e0a875f0-kube-api-access-mfj7j\") pod \"kuadrant-operator-controller-manager-68cf589589-bkf7x\" (UID: \"66eb3ae5-5efe-4070-a999-d399e0a875f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x"
Aug 05 17:55:21.785579 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:21.785524    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x"
Aug 05 17:55:21.926081 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:21.926059    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x"]
Aug 05 17:55:21.927909 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:55:21.927884    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66eb3ae5_5efe_4070_a999_d399e0a875f0.slice/crio-d752966dc9429b0f28c07da9013ca129e3ec4068a383c1e9b4beeb018b549e28 WatchSource:0}: Error finding container d752966dc9429b0f28c07da9013ca129e3ec4068a383c1e9b4beeb018b549e28: Status 404 returned error can't find the container with id d752966dc9429b0f28c07da9013ca129e3ec4068a383c1e9b4beeb018b549e28
Aug 05 17:55:22.777316 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:22.777274    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x" event={"ID":"66eb3ae5-5efe-4070-a999-d399e0a875f0","Type":"ContainerStarted","Data":"b4dd36160c6884d19da84288cb730f2273d3b7ca0974d7d47296084893bc1373"}
Aug 05 17:55:22.777791 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:22.777335    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x" event={"ID":"66eb3ae5-5efe-4070-a999-d399e0a875f0","Type":"ContainerStarted","Data":"d752966dc9429b0f28c07da9013ca129e3ec4068a383c1e9b4beeb018b549e28"}
Aug 05 17:55:22.777791 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:22.777403    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x"
Aug 05 17:55:22.867349 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:22.867289    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x" podStartSLOduration=1.867269345 podStartE2EDuration="1.867269345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:55:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:55:22.857550471 +0000 UTC m=+1330.210225076" watchObservedRunningTime="2026-08-05 17:55:22.867269345 +0000 UTC m=+1330.219943941"
Aug 05 17:55:33.785541 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:55:33.785507    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x"
Aug 05 17:57:11.299350 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:57:11.299271    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:58:07.504856 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:07.504819    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:58:17.477675 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:17.477637    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:58:39.698172 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:39.698090    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:58:49.528017 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.527986    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"]
Aug 05 17:58:49.531675 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.531655    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.534489 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.534467    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-7dd08526-kserve-self-signed-certs\""
Aug 05 17:58:49.542312 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.542292    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"]
Aug 05 17:58:49.635874 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.635842    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:58:49.729684 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.729656    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6jfv\" (UniqueName: \"kubernetes.io/projected/48b39382-bf56-47c7-827c-cc75a1cc524f-kube-api-access-l6jfv\") pod \"e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") " pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.729831 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.729708    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48b39382-bf56-47c7-827c-cc75a1cc524f-tls-certs\") pod \"e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") " pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.729831 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.729770    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-tmp-dir\") pod \"e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") " pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.729831 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.729821    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-home\") pod \"e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") " pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.730020 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.729857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-dshm\") pod \"e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") " pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.730020 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.729933    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-model-cache\") pod \"e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") " pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.830660 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.830628    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6jfv\" (UniqueName: \"kubernetes.io/projected/48b39382-bf56-47c7-827c-cc75a1cc524f-kube-api-access-l6jfv\") pod \"e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") " pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.830845 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.830670    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48b39382-bf56-47c7-827c-cc75a1cc524f-tls-certs\") pod \"e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") " pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.830845 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.830694    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-tmp-dir\") pod \"e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") " pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.830845 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.830722    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-home\") pod \"e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") " pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.830845 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.830746    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-dshm\") pod \"e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") " pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.830845 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.830787    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-model-cache\") pod \"e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") " pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.831173 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.831154    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-model-cache\") pod \"e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") " pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.831302 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.831269    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-home\") pod \"e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") " pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.831411 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.831303    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-tmp-dir\") pod \"e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") " pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.833528 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.833498    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48b39382-bf56-47c7-827c-cc75a1cc524f-tls-certs\") pod \"e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") " pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.833612 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.833536    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-dshm\") pod \"e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") " pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.839740 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.839714    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6jfv\" (UniqueName: \"kubernetes.io/projected/48b39382-bf56-47c7-827c-cc75a1cc524f-kube-api-access-l6jfv\") pod \"e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") " pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.840591 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.840569    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:49.988420 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:49.988285    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"]
Aug 05 17:58:49.990849 ip-10-0-131-77 kubenswrapper[2569]: W0805 17:58:49.990824    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48b39382_bf56_47c7_827c_cc75a1cc524f.slice/crio-9872526c0444fe6ff35aa28e46e2178b897d87f21391f1e2ea30f848a7d83cdf WatchSource:0}: Error finding container 9872526c0444fe6ff35aa28e46e2178b897d87f21391f1e2ea30f848a7d83cdf: Status 404 returned error can't find the container with id 9872526c0444fe6ff35aa28e46e2178b897d87f21391f1e2ea30f848a7d83cdf
Aug 05 17:58:50.824332 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:50.824300    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h" event={"ID":"48b39382-bf56-47c7-827c-cc75a1cc524f","Type":"ContainerStarted","Data":"75e2b74a343de2797f7808fa6fa0c1911e819427bb531f4816819d21212c4109"}
Aug 05 17:58:50.824332 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:50.824337    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h" event={"ID":"48b39382-bf56-47c7-827c-cc75a1cc524f","Type":"ContainerStarted","Data":"9872526c0444fe6ff35aa28e46e2178b897d87f21391f1e2ea30f848a7d83cdf"}
Aug 05 17:58:50.848328 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:50.848276    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h" podStartSLOduration=1.848260481 podStartE2EDuration="1.848260481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:58:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:58:50.845981282 +0000 UTC m=+1538.198655877" watchObservedRunningTime="2026-08-05 17:58:50.848260481 +0000 UTC m=+1538.200935101"
Aug 05 17:58:59.841366 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:59.841313    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:59.841795 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:59.841386    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:59.853743 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:59.853720    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:58:59.883541 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:58:59.883512    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:59:27.877630 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:27.877586    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:29.142235 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:29.142204    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"]
Aug 05 17:59:29.166569 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:29.166541    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:44.163796 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.163720    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h" podUID="48b39382-bf56-47c7-827c-cc75a1cc524f" containerName="main" containerID="cri-o://75e2b74a343de2797f7808fa6fa0c1911e819427bb531f4816819d21212c4109" gracePeriod=45
Aug 05 17:59:44.433402 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.433380    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:59:44.494750 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.494723    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-tmp-dir\") pod \"48b39382-bf56-47c7-827c-cc75a1cc524f\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") "
Aug 05 17:59:44.494886 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.494760    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-home\") pod \"48b39382-bf56-47c7-827c-cc75a1cc524f\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") "
Aug 05 17:59:44.494886 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.494796    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48b39382-bf56-47c7-827c-cc75a1cc524f-tls-certs\") pod \"48b39382-bf56-47c7-827c-cc75a1cc524f\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") "
Aug 05 17:59:44.494886 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.494867    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6jfv\" (UniqueName: \"kubernetes.io/projected/48b39382-bf56-47c7-827c-cc75a1cc524f-kube-api-access-l6jfv\") pod \"48b39382-bf56-47c7-827c-cc75a1cc524f\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") "
Aug 05 17:59:44.495036 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.494905    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-dshm\") pod \"48b39382-bf56-47c7-827c-cc75a1cc524f\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") "
Aug 05 17:59:44.495036 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.494968    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-model-cache\") pod \"48b39382-bf56-47c7-827c-cc75a1cc524f\" (UID: \"48b39382-bf56-47c7-827c-cc75a1cc524f\") "
Aug 05 17:59:44.495036 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.494988    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "48b39382-bf56-47c7-827c-cc75a1cc524f" (UID: "48b39382-bf56-47c7-827c-cc75a1cc524f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:59:44.495036 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.495007    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-home" (OuterVolumeSpecName: "home") pod "48b39382-bf56-47c7-827c-cc75a1cc524f" (UID: "48b39382-bf56-47c7-827c-cc75a1cc524f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:59:44.495282 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.495259    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-model-cache" (OuterVolumeSpecName: "model-cache") pod "48b39382-bf56-47c7-827c-cc75a1cc524f" (UID: "48b39382-bf56-47c7-827c-cc75a1cc524f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:59:44.495334 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.495305    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-tmp-dir\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:59:44.495334 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.495322    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-home\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:59:44.497106 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.497078    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48b39382-bf56-47c7-827c-cc75a1cc524f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "48b39382-bf56-47c7-827c-cc75a1cc524f" (UID: "48b39382-bf56-47c7-827c-cc75a1cc524f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:59:44.497255 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.497232    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-dshm" (OuterVolumeSpecName: "dshm") pod "48b39382-bf56-47c7-827c-cc75a1cc524f" (UID: "48b39382-bf56-47c7-827c-cc75a1cc524f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:59:44.497331 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.497304    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48b39382-bf56-47c7-827c-cc75a1cc524f-kube-api-access-l6jfv" (OuterVolumeSpecName: "kube-api-access-l6jfv") pod "48b39382-bf56-47c7-827c-cc75a1cc524f" (UID: "48b39382-bf56-47c7-827c-cc75a1cc524f"). InnerVolumeSpecName "kube-api-access-l6jfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:59:44.596608 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.596575    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48b39382-bf56-47c7-827c-cc75a1cc524f-tls-certs\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:59:44.596608 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.596604    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6jfv\" (UniqueName: \"kubernetes.io/projected/48b39382-bf56-47c7-827c-cc75a1cc524f-kube-api-access-l6jfv\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:59:44.596608 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.596613    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-dshm\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:59:44.596804 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:44.596622    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48b39382-bf56-47c7-827c-cc75a1cc524f-model-cache\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 17:59:45.107854 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:45.107817    2569 generic.go:356] "Generic (PLEG): container finished" podID="48b39382-bf56-47c7-827c-cc75a1cc524f" containerID="75e2b74a343de2797f7808fa6fa0c1911e819427bb531f4816819d21212c4109" exitCode=0
Aug 05 17:59:45.108047 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:45.107866    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"
Aug 05 17:59:45.108047 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:45.107884    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h" event={"ID":"48b39382-bf56-47c7-827c-cc75a1cc524f","Type":"ContainerDied","Data":"75e2b74a343de2797f7808fa6fa0c1911e819427bb531f4816819d21212c4109"}
Aug 05 17:59:45.108047 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:45.107923    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h" event={"ID":"48b39382-bf56-47c7-827c-cc75a1cc524f","Type":"ContainerDied","Data":"9872526c0444fe6ff35aa28e46e2178b897d87f21391f1e2ea30f848a7d83cdf"}
Aug 05 17:59:45.108047 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:45.107943    2569 scope.go:117] "RemoveContainer" containerID="75e2b74a343de2797f7808fa6fa0c1911e819427bb531f4816819d21212c4109"
Aug 05 17:59:45.118088 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:45.118072    2569 scope.go:117] "RemoveContainer" containerID="75e2b74a343de2797f7808fa6fa0c1911e819427bb531f4816819d21212c4109"
Aug 05 17:59:45.118394 ip-10-0-131-77 kubenswrapper[2569]: E0805 17:59:45.118373    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75e2b74a343de2797f7808fa6fa0c1911e819427bb531f4816819d21212c4109\": container with ID starting with 75e2b74a343de2797f7808fa6fa0c1911e819427bb531f4816819d21212c4109 not found: ID does not exist" containerID="75e2b74a343de2797f7808fa6fa0c1911e819427bb531f4816819d21212c4109"
Aug 05 17:59:45.118454 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:45.118402    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75e2b74a343de2797f7808fa6fa0c1911e819427bb531f4816819d21212c4109"} err="failed to get container status \"75e2b74a343de2797f7808fa6fa0c1911e819427bb531f4816819d21212c4109\": rpc error: code = NotFound desc = could not find container \"75e2b74a343de2797f7808fa6fa0c1911e819427bb531f4816819d21212c4109\": container with ID starting with 75e2b74a343de2797f7808fa6fa0c1911e819427bb531f4816819d21212c4109 not found: ID does not exist"
Aug 05 17:59:45.151940 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:45.151915    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"]
Aug 05 17:59:45.159222 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:45.159166    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-7dd08526-kserve-5886f545f9-ssw8h"]
Aug 05 17:59:47.150288 ip-10-0-131-77 kubenswrapper[2569]: I0805 17:59:47.150253    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48b39382-bf56-47c7-827c-cc75a1cc524f" path="/var/lib/kubelet/pods/48b39382-bf56-47c7-827c-cc75a1cc524f/volumes"
Aug 05 18:02:01.995681 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:02:01.995647    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:02:47.511388 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:02:47.511349    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:02:55.192316 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:02:55.192278    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:21.384811 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:21.384778    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:24.682348 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:24.682316    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:24.812818 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:24.812782    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:25.218214 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:25.218165    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:32.810735 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:32.810703    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:32.910000 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:32.909970    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:33.095744 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:33.095716    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:33.512248 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:33.512105    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:33.868338 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:33.868308    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:39.843314 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:39.843277    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:39.985502 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:39.985471    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:41.020612 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.020577    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"]
Aug 05 18:03:41.021034 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.021020    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48b39382-bf56-47c7-827c-cc75a1cc524f" containerName="main"
Aug 05 18:03:41.021077 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.021036    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="48b39382-bf56-47c7-827c-cc75a1cc524f" containerName="main"
Aug 05 18:03:41.021113 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.021096    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="48b39382-bf56-47c7-827c-cc75a1cc524f" containerName="main"
Aug 05 18:03:41.024371 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.024357    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"
Aug 05 18:03:41.027341 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.027318    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-8effb523-serving-cert\""
Aug 05 18:03:41.027462 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.027341    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jdhhr\""
Aug 05 18:03:41.057270 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.057243    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"]
Aug 05 18:03:41.088764 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.088737    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:41.130058 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.130028    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/012ed68a-fe8e-42e3-8ea2-bd6d516b1ded-openshift-service-ca\") pod \"maas-api-e2e-ait-8effb523-548d564fcd-f8p8k\" (UID: \"012ed68a-fe8e-42e3-8ea2-bd6d516b1ded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"
Aug 05 18:03:41.130214 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.130083    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dvgm\" (UniqueName: \"kubernetes.io/projected/012ed68a-fe8e-42e3-8ea2-bd6d516b1ded-kube-api-access-2dvgm\") pod \"maas-api-e2e-ait-8effb523-548d564fcd-f8p8k\" (UID: \"012ed68a-fe8e-42e3-8ea2-bd6d516b1ded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"
Aug 05 18:03:41.130214 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.130155    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/012ed68a-fe8e-42e3-8ea2-bd6d516b1ded-maas-api-tls\") pod \"maas-api-e2e-ait-8effb523-548d564fcd-f8p8k\" (UID: \"012ed68a-fe8e-42e3-8ea2-bd6d516b1ded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"
Aug 05 18:03:41.220741 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.220694    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:41.231264 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.231243    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/012ed68a-fe8e-42e3-8ea2-bd6d516b1ded-openshift-service-ca\") pod \"maas-api-e2e-ait-8effb523-548d564fcd-f8p8k\" (UID: \"012ed68a-fe8e-42e3-8ea2-bd6d516b1ded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"
Aug 05 18:03:41.231372 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.231307    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dvgm\" (UniqueName: \"kubernetes.io/projected/012ed68a-fe8e-42e3-8ea2-bd6d516b1ded-kube-api-access-2dvgm\") pod \"maas-api-e2e-ait-8effb523-548d564fcd-f8p8k\" (UID: \"012ed68a-fe8e-42e3-8ea2-bd6d516b1ded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"
Aug 05 18:03:41.231372 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.231365    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/012ed68a-fe8e-42e3-8ea2-bd6d516b1ded-maas-api-tls\") pod \"maas-api-e2e-ait-8effb523-548d564fcd-f8p8k\" (UID: \"012ed68a-fe8e-42e3-8ea2-bd6d516b1ded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"
Aug 05 18:03:41.231942 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.231925    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/012ed68a-fe8e-42e3-8ea2-bd6d516b1ded-openshift-service-ca\") pod \"maas-api-e2e-ait-8effb523-548d564fcd-f8p8k\" (UID: \"012ed68a-fe8e-42e3-8ea2-bd6d516b1ded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"
Aug 05 18:03:41.233882 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.233861    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/012ed68a-fe8e-42e3-8ea2-bd6d516b1ded-maas-api-tls\") pod \"maas-api-e2e-ait-8effb523-548d564fcd-f8p8k\" (UID: \"012ed68a-fe8e-42e3-8ea2-bd6d516b1ded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"
Aug 05 18:03:41.243920 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.243898    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dvgm\" (UniqueName: \"kubernetes.io/projected/012ed68a-fe8e-42e3-8ea2-bd6d516b1ded-kube-api-access-2dvgm\") pod \"maas-api-e2e-ait-8effb523-548d564fcd-f8p8k\" (UID: \"012ed68a-fe8e-42e3-8ea2-bd6d516b1ded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"
Aug 05 18:03:41.334170 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.334142    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"
Aug 05 18:03:41.693455 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.693423    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"]
Aug 05 18:03:41.694691 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:03:41.694664    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod012ed68a_fe8e_42e3_8ea2_bd6d516b1ded.slice/crio-92f6f2bfea71c401cca08e42fb8df8ddd79988375c4e5bac42f14ec4f82d3c98 WatchSource:0}: Error finding container 92f6f2bfea71c401cca08e42fb8df8ddd79988375c4e5bac42f14ec4f82d3c98: Status 404 returned error can't find the container with id 92f6f2bfea71c401cca08e42fb8df8ddd79988375c4e5bac42f14ec4f82d3c98
Aug 05 18:03:41.696513 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:41.696495    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 18:03:42.056793 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.056706    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:42.160487 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.160454    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:42.293534 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.293501    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw"]
Aug 05 18:03:42.298345 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.298329    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw"
Aug 05 18:03:42.305842 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.305800    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-8effb523\""
Aug 05 18:03:42.306102 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.306079    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-8effb523-dockercfg-z54fq\""
Aug 05 18:03:42.320373 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.320349    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw"]
Aug 05 18:03:42.322458 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.322436    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp"]
Aug 05 18:03:42.326818 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.326800    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp"
Aug 05 18:03:42.326993 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.326971    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k" event={"ID":"012ed68a-fe8e-42e3-8ea2-bd6d516b1ded","Type":"ContainerStarted","Data":"92f6f2bfea71c401cca08e42fb8df8ddd79988375c4e5bac42f14ec4f82d3c98"}
Aug 05 18:03:42.342107 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.342080    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba77f30d-0b54-4127-a089-2e65c90b90f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw\" (UID: \"ba77f30d-0b54-4127-a089-2e65c90b90f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw"
Aug 05 18:03:42.342236 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.342218    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6xhl\" (UniqueName: \"kubernetes.io/projected/ba77f30d-0b54-4127-a089-2e65c90b90f1-kube-api-access-g6xhl\") pod \"payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw\" (UID: \"ba77f30d-0b54-4127-a089-2e65c90b90f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw"
Aug 05 18:03:42.352768 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.352745    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp"]
Aug 05 18:03:42.443243 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.443148    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba77f30d-0b54-4127-a089-2e65c90b90f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw\" (UID: \"ba77f30d-0b54-4127-a089-2e65c90b90f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw"
Aug 05 18:03:42.443421 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.443269    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56wmf\" (UniqueName: \"kubernetes.io/projected/0b0cb04a-b2f4-44cf-a57c-afba0d2fe534-kube-api-access-56wmf\") pod \"payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp\" (UID: \"0b0cb04a-b2f4-44cf-a57c-afba0d2fe534\") " pod="openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp"
Aug 05 18:03:42.443421 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.443309    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0cb04a-b2f4-44cf-a57c-afba0d2fe534-plugins-config-volume\") pod \"payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp\" (UID: \"0b0cb04a-b2f4-44cf-a57c-afba0d2fe534\") " pod="openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp"
Aug 05 18:03:42.443421 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.443344    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6xhl\" (UniqueName: \"kubernetes.io/projected/ba77f30d-0b54-4127-a089-2e65c90b90f1-kube-api-access-g6xhl\") pod \"payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw\" (UID: \"ba77f30d-0b54-4127-a089-2e65c90b90f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw"
Aug 05 18:03:42.444029 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.444003    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba77f30d-0b54-4127-a089-2e65c90b90f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw\" (UID: \"ba77f30d-0b54-4127-a089-2e65c90b90f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw"
Aug 05 18:03:42.458345 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.458316    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6xhl\" (UniqueName: \"kubernetes.io/projected/ba77f30d-0b54-4127-a089-2e65c90b90f1-kube-api-access-g6xhl\") pod \"payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw\" (UID: \"ba77f30d-0b54-4127-a089-2e65c90b90f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw"
Aug 05 18:03:42.544861 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.544837    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56wmf\" (UniqueName: \"kubernetes.io/projected/0b0cb04a-b2f4-44cf-a57c-afba0d2fe534-kube-api-access-56wmf\") pod \"payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp\" (UID: \"0b0cb04a-b2f4-44cf-a57c-afba0d2fe534\") " pod="openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp"
Aug 05 18:03:42.544979 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.544874    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0cb04a-b2f4-44cf-a57c-afba0d2fe534-plugins-config-volume\") pod \"payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp\" (UID: \"0b0cb04a-b2f4-44cf-a57c-afba0d2fe534\") " pod="openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp"
Aug 05 18:03:42.545534 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.545517    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0cb04a-b2f4-44cf-a57c-afba0d2fe534-plugins-config-volume\") pod \"payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp\" (UID: \"0b0cb04a-b2f4-44cf-a57c-afba0d2fe534\") " pod="openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp"
Aug 05 18:03:42.563970 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.563943    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56wmf\" (UniqueName: \"kubernetes.io/projected/0b0cb04a-b2f4-44cf-a57c-afba0d2fe534-kube-api-access-56wmf\") pod \"payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp\" (UID: \"0b0cb04a-b2f4-44cf-a57c-afba0d2fe534\") " pod="openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp"
Aug 05 18:03:42.609894 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.609823    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw"
Aug 05 18:03:42.638153 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.638127    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp"
Aug 05 18:03:42.776886 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.776843    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw"]
Aug 05 18:03:42.780918 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:03:42.780889    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba77f30d_0b54_4127_a089_2e65c90b90f1.slice/crio-298c77e63b41a56d6d5d7de50ede0528f48cd9c5566c815846fa1931162829f2 WatchSource:0}: Error finding container 298c77e63b41a56d6d5d7de50ede0528f48cd9c5566c815846fa1931162829f2: Status 404 returned error can't find the container with id 298c77e63b41a56d6d5d7de50ede0528f48cd9c5566c815846fa1931162829f2
Aug 05 18:03:42.817803 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:42.817778    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp"]
Aug 05 18:03:42.818844 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:03:42.818821    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b0cb04a_b2f4_44cf_a57c_afba0d2fe534.slice/crio-6ee2fc25f1db5d87eccf5316c6ca95f9477aec0dfb07dac0804cae362c91015e WatchSource:0}: Error finding container 6ee2fc25f1db5d87eccf5316c6ca95f9477aec0dfb07dac0804cae362c91015e: Status 404 returned error can't find the container with id 6ee2fc25f1db5d87eccf5316c6ca95f9477aec0dfb07dac0804cae362c91015e
Aug 05 18:03:43.333454 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:43.333422    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp" event={"ID":"0b0cb04a-b2f4-44cf-a57c-afba0d2fe534","Type":"ContainerStarted","Data":"454e8a68dec1cbdf440d2c2f8012139bbf17a56c33dfe467d6b2a1500d09d8b4"}
Aug 05 18:03:43.333917 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:43.333481    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp" event={"ID":"0b0cb04a-b2f4-44cf-a57c-afba0d2fe534","Type":"ContainerStarted","Data":"6ee2fc25f1db5d87eccf5316c6ca95f9477aec0dfb07dac0804cae362c91015e"}
Aug 05 18:03:43.333917 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:43.333541    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp"
Aug 05 18:03:43.335080 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:43.335052    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw" event={"ID":"ba77f30d-0b54-4127-a089-2e65c90b90f1","Type":"ContainerStarted","Data":"be01267d5d4f21e19b2493adc57a8393fe9e4affcc829471001053d7915a7863"}
Aug 05 18:03:43.335226 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:43.335105    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw"
Aug 05 18:03:43.335226 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:43.335122    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw" event={"ID":"ba77f30d-0b54-4127-a089-2e65c90b90f1","Type":"ContainerStarted","Data":"298c77e63b41a56d6d5d7de50ede0528f48cd9c5566c815846fa1931162829f2"}
Aug 05 18:03:43.336505 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:43.336483    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k" event={"ID":"012ed68a-fe8e-42e3-8ea2-bd6d516b1ded","Type":"ContainerStarted","Data":"f13b54f59f8eb06d76bb7ef47c1aec8c37b2304fcef5352d87e5c13935b3871d"}
Aug 05 18:03:43.336625 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:43.336591    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"
Aug 05 18:03:43.447876 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:43.447824    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp" podStartSLOduration=1.447807958 podStartE2EDuration="1.447807958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:03:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:03:43.446662956 +0000 UTC m=+1830.799337551" watchObservedRunningTime="2026-08-05 18:03:43.447807958 +0000 UTC m=+1830.800482552"
Aug 05 18:03:43.630069 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:43.629949    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k" podStartSLOduration=2.855555186 podStartE2EDuration="3.629924982s" totalImagesPullingTime="774.369796ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:03:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:03:43.548029779 +0000 UTC m=+1830.900704372" watchObservedRunningTime="2026-08-05 18:03:43.629924982 +0000 UTC m=+1830.982599575"
Aug 05 18:03:49.348526 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:49.348497    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"
Aug 05 18:03:49.383641 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:49.383593    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw" podStartSLOduration=7.383578276 podStartE2EDuration="7.383578276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:03:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:03:43.632407543 +0000 UTC m=+1830.985082137" watchObservedRunningTime="2026-08-05 18:03:49.383578276 +0000 UTC m=+1836.736252869"
Aug 05 18:03:54.343975 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:54.343947    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp"
Aug 05 18:03:54.344479 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:03:54.344024    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw"
Aug 05 18:04:21.851592 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:21.851562    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"]
Aug 05 18:04:21.852092 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:21.851749    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k" podUID="012ed68a-fe8e-42e3-8ea2-bd6d516b1ded" containerName="maas-api" containerID="cri-o://f13b54f59f8eb06d76bb7ef47c1aec8c37b2304fcef5352d87e5c13935b3871d" gracePeriod=30
Aug 05 18:04:21.887130 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:21.887101    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:21.947322 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:21.947291    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp"]
Aug 05 18:04:21.947595 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:21.947550    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp" podUID="0b0cb04a-b2f4-44cf-a57c-afba0d2fe534" containerName="payload-processing" containerID="cri-o://454e8a68dec1cbdf440d2c2f8012139bbf17a56c33dfe467d6b2a1500d09d8b4" gracePeriod=30
Aug 05 18:04:21.957563 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:21.957525    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw"]
Aug 05 18:04:21.958526 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:21.957744    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw" podUID="ba77f30d-0b54-4127-a089-2e65c90b90f1" containerName="payload-pre-processing" containerID="cri-o://be01267d5d4f21e19b2493adc57a8393fe9e4affcc829471001053d7915a7863" gracePeriod=30
Aug 05 18:04:21.999420 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:21.999387    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:22.112037 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.111936    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:22.145466 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.145444    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"
Aug 05 18:04:22.206858 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.206831    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp"
Aug 05 18:04:22.212063 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.212037    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/012ed68a-fe8e-42e3-8ea2-bd6d516b1ded-openshift-service-ca\") pod \"012ed68a-fe8e-42e3-8ea2-bd6d516b1ded\" (UID: \"012ed68a-fe8e-42e3-8ea2-bd6d516b1ded\") "
Aug 05 18:04:22.212239 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.212080    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/012ed68a-fe8e-42e3-8ea2-bd6d516b1ded-maas-api-tls\") pod \"012ed68a-fe8e-42e3-8ea2-bd6d516b1ded\" (UID: \"012ed68a-fe8e-42e3-8ea2-bd6d516b1ded\") "
Aug 05 18:04:22.212321 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.212271    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dvgm\" (UniqueName: \"kubernetes.io/projected/012ed68a-fe8e-42e3-8ea2-bd6d516b1ded-kube-api-access-2dvgm\") pod \"012ed68a-fe8e-42e3-8ea2-bd6d516b1ded\" (UID: \"012ed68a-fe8e-42e3-8ea2-bd6d516b1ded\") "
Aug 05 18:04:22.212544 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.212503    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/012ed68a-fe8e-42e3-8ea2-bd6d516b1ded-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "012ed68a-fe8e-42e3-8ea2-bd6d516b1ded" (UID: "012ed68a-fe8e-42e3-8ea2-bd6d516b1ded"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:04:22.212711 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.212688    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/012ed68a-fe8e-42e3-8ea2-bd6d516b1ded-openshift-service-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:04:22.215118 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.215093    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/012ed68a-fe8e-42e3-8ea2-bd6d516b1ded-kube-api-access-2dvgm" (OuterVolumeSpecName: "kube-api-access-2dvgm") pod "012ed68a-fe8e-42e3-8ea2-bd6d516b1ded" (UID: "012ed68a-fe8e-42e3-8ea2-bd6d516b1ded"). InnerVolumeSpecName "kube-api-access-2dvgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:04:22.215267 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.215149    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/012ed68a-fe8e-42e3-8ea2-bd6d516b1ded-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "012ed68a-fe8e-42e3-8ea2-bd6d516b1ded" (UID: "012ed68a-fe8e-42e3-8ea2-bd6d516b1ded"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:04:22.232661 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.232632    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:22.244902 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.244881    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw"
Aug 05 18:04:22.313790 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.313755    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0cb04a-b2f4-44cf-a57c-afba0d2fe534-plugins-config-volume\") pod \"0b0cb04a-b2f4-44cf-a57c-afba0d2fe534\" (UID: \"0b0cb04a-b2f4-44cf-a57c-afba0d2fe534\") "
Aug 05 18:04:22.313943 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.313867    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6xhl\" (UniqueName: \"kubernetes.io/projected/ba77f30d-0b54-4127-a089-2e65c90b90f1-kube-api-access-g6xhl\") pod \"ba77f30d-0b54-4127-a089-2e65c90b90f1\" (UID: \"ba77f30d-0b54-4127-a089-2e65c90b90f1\") "
Aug 05 18:04:22.313943 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.313906    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56wmf\" (UniqueName: \"kubernetes.io/projected/0b0cb04a-b2f4-44cf-a57c-afba0d2fe534-kube-api-access-56wmf\") pod \"0b0cb04a-b2f4-44cf-a57c-afba0d2fe534\" (UID: \"0b0cb04a-b2f4-44cf-a57c-afba0d2fe534\") "
Aug 05 18:04:22.314069 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.313968    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba77f30d-0b54-4127-a089-2e65c90b90f1-plugins-config-volume\") pod \"ba77f30d-0b54-4127-a089-2e65c90b90f1\" (UID: \"ba77f30d-0b54-4127-a089-2e65c90b90f1\") "
Aug 05 18:04:22.314207 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.314142    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b0cb04a-b2f4-44cf-a57c-afba0d2fe534-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0b0cb04a-b2f4-44cf-a57c-afba0d2fe534" (UID: "0b0cb04a-b2f4-44cf-a57c-afba0d2fe534"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:04:22.314333 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.314312    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dvgm\" (UniqueName: \"kubernetes.io/projected/012ed68a-fe8e-42e3-8ea2-bd6d516b1ded-kube-api-access-2dvgm\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:04:22.314485 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.314472    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0cb04a-b2f4-44cf-a57c-afba0d2fe534-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:04:22.314577 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.314566    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/012ed68a-fe8e-42e3-8ea2-bd6d516b1ded-maas-api-tls\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:04:22.314659 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.314573    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba77f30d-0b54-4127-a089-2e65c90b90f1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ba77f30d-0b54-4127-a089-2e65c90b90f1" (UID: "ba77f30d-0b54-4127-a089-2e65c90b90f1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:04:22.317235 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.317206    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b0cb04a-b2f4-44cf-a57c-afba0d2fe534-kube-api-access-56wmf" (OuterVolumeSpecName: "kube-api-access-56wmf") pod "0b0cb04a-b2f4-44cf-a57c-afba0d2fe534" (UID: "0b0cb04a-b2f4-44cf-a57c-afba0d2fe534"). InnerVolumeSpecName "kube-api-access-56wmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:04:22.317235 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.317237    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba77f30d-0b54-4127-a089-2e65c90b90f1-kube-api-access-g6xhl" (OuterVolumeSpecName: "kube-api-access-g6xhl") pod "ba77f30d-0b54-4127-a089-2e65c90b90f1" (UID: "ba77f30d-0b54-4127-a089-2e65c90b90f1"). InnerVolumeSpecName "kube-api-access-g6xhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:04:22.415477 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.415402    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6xhl\" (UniqueName: \"kubernetes.io/projected/ba77f30d-0b54-4127-a089-2e65c90b90f1-kube-api-access-g6xhl\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:04:22.415477 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.415429    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56wmf\" (UniqueName: \"kubernetes.io/projected/0b0cb04a-b2f4-44cf-a57c-afba0d2fe534-kube-api-access-56wmf\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:04:22.415477 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.415440    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba77f30d-0b54-4127-a089-2e65c90b90f1-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:04:22.486520 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.486489    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:22.543604 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.543573    2569 generic.go:356] "Generic (PLEG): container finished" podID="012ed68a-fe8e-42e3-8ea2-bd6d516b1ded" containerID="f13b54f59f8eb06d76bb7ef47c1aec8c37b2304fcef5352d87e5c13935b3871d" exitCode=0
Aug 05 18:04:22.543784 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.543623    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"
Aug 05 18:04:22.543784 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.543669    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k" event={"ID":"012ed68a-fe8e-42e3-8ea2-bd6d516b1ded","Type":"ContainerDied","Data":"f13b54f59f8eb06d76bb7ef47c1aec8c37b2304fcef5352d87e5c13935b3871d"}
Aug 05 18:04:22.543784 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.543735    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k" event={"ID":"012ed68a-fe8e-42e3-8ea2-bd6d516b1ded","Type":"ContainerDied","Data":"92f6f2bfea71c401cca08e42fb8df8ddd79988375c4e5bac42f14ec4f82d3c98"}
Aug 05 18:04:22.543784 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.543766    2569 scope.go:117] "RemoveContainer" containerID="f13b54f59f8eb06d76bb7ef47c1aec8c37b2304fcef5352d87e5c13935b3871d"
Aug 05 18:04:22.545209 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.545157    2569 generic.go:356] "Generic (PLEG): container finished" podID="0b0cb04a-b2f4-44cf-a57c-afba0d2fe534" containerID="454e8a68dec1cbdf440d2c2f8012139bbf17a56c33dfe467d6b2a1500d09d8b4" exitCode=0
Aug 05 18:04:22.545336 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.545269    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp" event={"ID":"0b0cb04a-b2f4-44cf-a57c-afba0d2fe534","Type":"ContainerDied","Data":"454e8a68dec1cbdf440d2c2f8012139bbf17a56c33dfe467d6b2a1500d09d8b4"}
Aug 05 18:04:22.545336 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.545267    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp"
Aug 05 18:04:22.545336 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.545313    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp" event={"ID":"0b0cb04a-b2f4-44cf-a57c-afba0d2fe534","Type":"ContainerDied","Data":"6ee2fc25f1db5d87eccf5316c6ca95f9477aec0dfb07dac0804cae362c91015e"}
Aug 05 18:04:22.546765 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.546743    2569 generic.go:356] "Generic (PLEG): container finished" podID="ba77f30d-0b54-4127-a089-2e65c90b90f1" containerID="be01267d5d4f21e19b2493adc57a8393fe9e4affcc829471001053d7915a7863" exitCode=0
Aug 05 18:04:22.546874 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.546779    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw" event={"ID":"ba77f30d-0b54-4127-a089-2e65c90b90f1","Type":"ContainerDied","Data":"be01267d5d4f21e19b2493adc57a8393fe9e4affcc829471001053d7915a7863"}
Aug 05 18:04:22.546874 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.546815    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw" event={"ID":"ba77f30d-0b54-4127-a089-2e65c90b90f1","Type":"ContainerDied","Data":"298c77e63b41a56d6d5d7de50ede0528f48cd9c5566c815846fa1931162829f2"}
Aug 05 18:04:22.546874 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.546819    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw"
Aug 05 18:04:22.554668 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.554647    2569 scope.go:117] "RemoveContainer" containerID="f13b54f59f8eb06d76bb7ef47c1aec8c37b2304fcef5352d87e5c13935b3871d"
Aug 05 18:04:22.554936 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:04:22.554916    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f13b54f59f8eb06d76bb7ef47c1aec8c37b2304fcef5352d87e5c13935b3871d\": container with ID starting with f13b54f59f8eb06d76bb7ef47c1aec8c37b2304fcef5352d87e5c13935b3871d not found: ID does not exist" containerID="f13b54f59f8eb06d76bb7ef47c1aec8c37b2304fcef5352d87e5c13935b3871d"
Aug 05 18:04:22.555002 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.554943    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f13b54f59f8eb06d76bb7ef47c1aec8c37b2304fcef5352d87e5c13935b3871d"} err="failed to get container status \"f13b54f59f8eb06d76bb7ef47c1aec8c37b2304fcef5352d87e5c13935b3871d\": rpc error: code = NotFound desc = could not find container \"f13b54f59f8eb06d76bb7ef47c1aec8c37b2304fcef5352d87e5c13935b3871d\": container with ID starting with f13b54f59f8eb06d76bb7ef47c1aec8c37b2304fcef5352d87e5c13935b3871d not found: ID does not exist"
Aug 05 18:04:22.555002 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.554961    2569 scope.go:117] "RemoveContainer" containerID="454e8a68dec1cbdf440d2c2f8012139bbf17a56c33dfe467d6b2a1500d09d8b4"
Aug 05 18:04:22.563158 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.563141    2569 scope.go:117] "RemoveContainer" containerID="454e8a68dec1cbdf440d2c2f8012139bbf17a56c33dfe467d6b2a1500d09d8b4"
Aug 05 18:04:22.563465 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:04:22.563447    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"454e8a68dec1cbdf440d2c2f8012139bbf17a56c33dfe467d6b2a1500d09d8b4\": container with ID starting with 454e8a68dec1cbdf440d2c2f8012139bbf17a56c33dfe467d6b2a1500d09d8b4 not found: ID does not exist" containerID="454e8a68dec1cbdf440d2c2f8012139bbf17a56c33dfe467d6b2a1500d09d8b4"
Aug 05 18:04:22.563512 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.563472    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"454e8a68dec1cbdf440d2c2f8012139bbf17a56c33dfe467d6b2a1500d09d8b4"} err="failed to get container status \"454e8a68dec1cbdf440d2c2f8012139bbf17a56c33dfe467d6b2a1500d09d8b4\": rpc error: code = NotFound desc = could not find container \"454e8a68dec1cbdf440d2c2f8012139bbf17a56c33dfe467d6b2a1500d09d8b4\": container with ID starting with 454e8a68dec1cbdf440d2c2f8012139bbf17a56c33dfe467d6b2a1500d09d8b4 not found: ID does not exist"
Aug 05 18:04:22.563512 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.563488    2569 scope.go:117] "RemoveContainer" containerID="be01267d5d4f21e19b2493adc57a8393fe9e4affcc829471001053d7915a7863"
Aug 05 18:04:22.571829 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.571813    2569 scope.go:117] "RemoveContainer" containerID="be01267d5d4f21e19b2493adc57a8393fe9e4affcc829471001053d7915a7863"
Aug 05 18:04:22.572073 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:04:22.572055    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be01267d5d4f21e19b2493adc57a8393fe9e4affcc829471001053d7915a7863\": container with ID starting with be01267d5d4f21e19b2493adc57a8393fe9e4affcc829471001053d7915a7863 not found: ID does not exist" containerID="be01267d5d4f21e19b2493adc57a8393fe9e4affcc829471001053d7915a7863"
Aug 05 18:04:22.572119 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.572082    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be01267d5d4f21e19b2493adc57a8393fe9e4affcc829471001053d7915a7863"} err="failed to get container status \"be01267d5d4f21e19b2493adc57a8393fe9e4affcc829471001053d7915a7863\": rpc error: code = NotFound desc = could not find container \"be01267d5d4f21e19b2493adc57a8393fe9e4affcc829471001053d7915a7863\": container with ID starting with be01267d5d4f21e19b2493adc57a8393fe9e4affcc829471001053d7915a7863 not found: ID does not exist"
Aug 05 18:04:22.577927 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.577905    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"]
Aug 05 18:04:22.580856 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.580837    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8effb523-548d564fcd-f8p8k"]
Aug 05 18:04:22.597282 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.597263    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp"]
Aug 05 18:04:22.598949 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.598931    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8effb523-f4c9548db-2lfbp"]
Aug 05 18:04:22.604477 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.604450    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:22.613298 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.613279    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw"]
Aug 05 18:04:22.620735 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.620713    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8effb523-75dcf8d7c6-zzzqw"]
Aug 05 18:04:22.900315 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:22.900285    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:23.042306 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:23.042269    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:23.152537 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:23.152458    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="012ed68a-fe8e-42e3-8ea2-bd6d516b1ded" path="/var/lib/kubelet/pods/012ed68a-fe8e-42e3-8ea2-bd6d516b1ded/volumes"
Aug 05 18:04:23.152873 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:23.152858    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b0cb04a-b2f4-44cf-a57c-afba0d2fe534" path="/var/lib/kubelet/pods/0b0cb04a-b2f4-44cf-a57c-afba0d2fe534/volumes"
Aug 05 18:04:23.153248 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:23.153235    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba77f30d-0b54-4127-a089-2e65c90b90f1" path="/var/lib/kubelet/pods/ba77f30d-0b54-4127-a089-2e65c90b90f1/volumes"
Aug 05 18:04:23.362679 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:23.362651    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:23.507322 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:23.507242    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:23.605526 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:23.605488    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:26.061985 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.061950    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:26.139063 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.139031    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"]
Aug 05 18:04:26.139541 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.139522    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b0cb04a-b2f4-44cf-a57c-afba0d2fe534" containerName="payload-processing"
Aug 05 18:04:26.139541 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.139540    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b0cb04a-b2f4-44cf-a57c-afba0d2fe534" containerName="payload-processing"
Aug 05 18:04:26.139739 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.139571    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="012ed68a-fe8e-42e3-8ea2-bd6d516b1ded" containerName="maas-api"
Aug 05 18:04:26.139739 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.139580    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="012ed68a-fe8e-42e3-8ea2-bd6d516b1ded" containerName="maas-api"
Aug 05 18:04:26.139739 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.139595    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba77f30d-0b54-4127-a089-2e65c90b90f1" containerName="payload-pre-processing"
Aug 05 18:04:26.139739 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.139603    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba77f30d-0b54-4127-a089-2e65c90b90f1" containerName="payload-pre-processing"
Aug 05 18:04:26.139739 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.139711    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b0cb04a-b2f4-44cf-a57c-afba0d2fe534" containerName="payload-processing"
Aug 05 18:04:26.139739 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.139725    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba77f30d-0b54-4127-a089-2e65c90b90f1" containerName="payload-pre-processing"
Aug 05 18:04:26.139739 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.139734    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="012ed68a-fe8e-42e3-8ea2-bd6d516b1ded" containerName="maas-api"
Aug 05 18:04:26.144819 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.144797    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"
Aug 05 18:04:26.147538 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.147517    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-e7e381dc-serving-cert\""
Aug 05 18:04:26.147800 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.147784    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jdhhr\""
Aug 05 18:04:26.155753 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.155732    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"]
Aug 05 18:04:26.172702 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.172675    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:26.253231 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.253170    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c75c11c-1c32-4b96-a38d-abc2ad3e8f99-maas-api-tls\") pod \"maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw\" (UID: \"9c75c11c-1c32-4b96-a38d-abc2ad3e8f99\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"
Aug 05 18:04:26.253375 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.253311    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g7q8\" (UniqueName: \"kubernetes.io/projected/9c75c11c-1c32-4b96-a38d-abc2ad3e8f99-kube-api-access-9g7q8\") pod \"maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw\" (UID: \"9c75c11c-1c32-4b96-a38d-abc2ad3e8f99\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"
Aug 05 18:04:26.253441 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.253425    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c75c11c-1c32-4b96-a38d-abc2ad3e8f99-openshift-service-ca\") pod \"maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw\" (UID: \"9c75c11c-1c32-4b96-a38d-abc2ad3e8f99\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"
Aug 05 18:04:26.354371 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.354339    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g7q8\" (UniqueName: \"kubernetes.io/projected/9c75c11c-1c32-4b96-a38d-abc2ad3e8f99-kube-api-access-9g7q8\") pod \"maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw\" (UID: \"9c75c11c-1c32-4b96-a38d-abc2ad3e8f99\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"
Aug 05 18:04:26.354545 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.354425    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c75c11c-1c32-4b96-a38d-abc2ad3e8f99-openshift-service-ca\") pod \"maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw\" (UID: \"9c75c11c-1c32-4b96-a38d-abc2ad3e8f99\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"
Aug 05 18:04:26.354545 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.354457    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c75c11c-1c32-4b96-a38d-abc2ad3e8f99-maas-api-tls\") pod \"maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw\" (UID: \"9c75c11c-1c32-4b96-a38d-abc2ad3e8f99\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"
Aug 05 18:04:26.355070 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.355048    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c75c11c-1c32-4b96-a38d-abc2ad3e8f99-openshift-service-ca\") pod \"maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw\" (UID: \"9c75c11c-1c32-4b96-a38d-abc2ad3e8f99\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"
Aug 05 18:04:26.356953 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.356934    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c75c11c-1c32-4b96-a38d-abc2ad3e8f99-maas-api-tls\") pod \"maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw\" (UID: \"9c75c11c-1c32-4b96-a38d-abc2ad3e8f99\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"
Aug 05 18:04:26.363455 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.363435    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g7q8\" (UniqueName: \"kubernetes.io/projected/9c75c11c-1c32-4b96-a38d-abc2ad3e8f99-kube-api-access-9g7q8\") pod \"maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw\" (UID: \"9c75c11c-1c32-4b96-a38d-abc2ad3e8f99\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"
Aug 05 18:04:26.378035 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.378008    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:26.457989 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.457969    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"
Aug 05 18:04:26.615904 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.615878    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"]
Aug 05 18:04:26.617575 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:04:26.617544    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c75c11c_1c32_4b96_a38d_abc2ad3e8f99.slice/crio-8752f28b625c1ffefdc7d69d1e65bc4ff4f09bd34ad74a4ff7f7965bb4d68250 WatchSource:0}: Error finding container 8752f28b625c1ffefdc7d69d1e65bc4ff4f09bd34ad74a4ff7f7965bb4d68250: Status 404 returned error can't find the container with id 8752f28b625c1ffefdc7d69d1e65bc4ff4f09bd34ad74a4ff7f7965bb4d68250
Aug 05 18:04:26.735409 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.735374    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:26.885083 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:26.884980    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:27.037541 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.037513    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv"]
Aug 05 18:04:27.042073 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.042053    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv"
Aug 05 18:04:27.045040 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.045017    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-e7e381dc\""
Aug 05 18:04:27.045277 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.045253    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-e7e381dc-dockercfg-zfbnj\""
Aug 05 18:04:27.053845 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.053809    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv"]
Aug 05 18:04:27.070506 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.070484    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s"]
Aug 05 18:04:27.074634 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.074617    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s"
Aug 05 18:04:27.085963 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.085937    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s"]
Aug 05 18:04:27.160988 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.160902    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlvd7\" (UniqueName: \"kubernetes.io/projected/0d934782-ebf8-4997-9625-0064a036b167-kube-api-access-wlvd7\") pod \"payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv\" (UID: \"0d934782-ebf8-4997-9625-0064a036b167\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv"
Aug 05 18:04:27.160988 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.160953    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d934782-ebf8-4997-9625-0064a036b167-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv\" (UID: \"0d934782-ebf8-4997-9625-0064a036b167\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv"
Aug 05 18:04:27.161167 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.161033    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ddf7bf1-7da7-4c58-a3b6-853320a7fbec-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s\" (UID: \"3ddf7bf1-7da7-4c58-a3b6-853320a7fbec\") " pod="openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s"
Aug 05 18:04:27.161167 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.161078    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnzz5\" (UniqueName: \"kubernetes.io/projected/3ddf7bf1-7da7-4c58-a3b6-853320a7fbec-kube-api-access-qnzz5\") pod \"payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s\" (UID: \"3ddf7bf1-7da7-4c58-a3b6-853320a7fbec\") " pod="openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s"
Aug 05 18:04:27.262208 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.262142    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlvd7\" (UniqueName: \"kubernetes.io/projected/0d934782-ebf8-4997-9625-0064a036b167-kube-api-access-wlvd7\") pod \"payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv\" (UID: \"0d934782-ebf8-4997-9625-0064a036b167\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv"
Aug 05 18:04:27.262380 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.262233    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d934782-ebf8-4997-9625-0064a036b167-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv\" (UID: \"0d934782-ebf8-4997-9625-0064a036b167\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv"
Aug 05 18:04:27.262380 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.262275    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ddf7bf1-7da7-4c58-a3b6-853320a7fbec-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s\" (UID: \"3ddf7bf1-7da7-4c58-a3b6-853320a7fbec\") " pod="openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s"
Aug 05 18:04:27.262380 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.262323    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnzz5\" (UniqueName: \"kubernetes.io/projected/3ddf7bf1-7da7-4c58-a3b6-853320a7fbec-kube-api-access-qnzz5\") pod \"payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s\" (UID: \"3ddf7bf1-7da7-4c58-a3b6-853320a7fbec\") " pod="openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s"
Aug 05 18:04:27.263072 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.263045    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d934782-ebf8-4997-9625-0064a036b167-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv\" (UID: \"0d934782-ebf8-4997-9625-0064a036b167\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv"
Aug 05 18:04:27.263243 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.263220    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ddf7bf1-7da7-4c58-a3b6-853320a7fbec-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s\" (UID: \"3ddf7bf1-7da7-4c58-a3b6-853320a7fbec\") " pod="openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s"
Aug 05 18:04:27.271434 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.271415    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnzz5\" (UniqueName: \"kubernetes.io/projected/3ddf7bf1-7da7-4c58-a3b6-853320a7fbec-kube-api-access-qnzz5\") pod \"payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s\" (UID: \"3ddf7bf1-7da7-4c58-a3b6-853320a7fbec\") " pod="openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s"
Aug 05 18:04:27.271620 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.271593    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlvd7\" (UniqueName: \"kubernetes.io/projected/0d934782-ebf8-4997-9625-0064a036b167-kube-api-access-wlvd7\") pod \"payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv\" (UID: \"0d934782-ebf8-4997-9625-0064a036b167\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv"
Aug 05 18:04:27.357641 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.357611    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv"
Aug 05 18:04:27.387044 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.387017    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s"
Aug 05 18:04:27.515159 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.515129    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv"]
Aug 05 18:04:27.518384 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:04:27.518342    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d934782_ebf8_4997_9625_0064a036b167.slice/crio-d66317f88264af3d5f17bd1fcba72d626553d123dc6d4158597fc5f29268e8e5 WatchSource:0}: Error finding container d66317f88264af3d5f17bd1fcba72d626553d123dc6d4158597fc5f29268e8e5: Status 404 returned error can't find the container with id d66317f88264af3d5f17bd1fcba72d626553d123dc6d4158597fc5f29268e8e5
Aug 05 18:04:27.549165 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.549143    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s"]
Aug 05 18:04:27.553401 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:04:27.553375    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ddf7bf1_7da7_4c58_a3b6_853320a7fbec.slice/crio-a8afb2ccf05758edb607e71500f9156a791897229d5460898a64c31e581a6a33 WatchSource:0}: Error finding container a8afb2ccf05758edb607e71500f9156a791897229d5460898a64c31e581a6a33: Status 404 returned error can't find the container with id a8afb2ccf05758edb607e71500f9156a791897229d5460898a64c31e581a6a33
Aug 05 18:04:27.583097 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.583060    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv" event={"ID":"0d934782-ebf8-4997-9625-0064a036b167","Type":"ContainerStarted","Data":"d66317f88264af3d5f17bd1fcba72d626553d123dc6d4158597fc5f29268e8e5"}
Aug 05 18:04:27.584820 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.584782    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s" event={"ID":"3ddf7bf1-7da7-4c58-a3b6-853320a7fbec","Type":"ContainerStarted","Data":"a8afb2ccf05758edb607e71500f9156a791897229d5460898a64c31e581a6a33"}
Aug 05 18:04:27.588468 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.587726    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw" event={"ID":"9c75c11c-1c32-4b96-a38d-abc2ad3e8f99","Type":"ContainerStarted","Data":"1011abe0e10944720f1e1f3c17a81d75f7b1ba0ab9feae131e57a7b9a50f11a8"}
Aug 05 18:04:27.588468 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.587778    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw" event={"ID":"9c75c11c-1c32-4b96-a38d-abc2ad3e8f99","Type":"ContainerStarted","Data":"8752f28b625c1ffefdc7d69d1e65bc4ff4f09bd34ad74a4ff7f7965bb4d68250"}
Aug 05 18:04:27.588468 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.588341    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"
Aug 05 18:04:27.612396 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:27.612352    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw" podStartSLOduration=1.221552251 podStartE2EDuration="1.612335563s" totalImagesPullingTime="390.783312ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:04:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:04:27.610993792 +0000 UTC m=+1874.963668387" watchObservedRunningTime="2026-08-05 18:04:27.612335563 +0000 UTC m=+1874.965010157"
Aug 05 18:04:28.594998 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:28.594958    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s" event={"ID":"3ddf7bf1-7da7-4c58-a3b6-853320a7fbec","Type":"ContainerStarted","Data":"cbb95d7865f6f6af952891a174e485d6aa5491c9b153c7a8b43cad88d6e35f05"}
Aug 05 18:04:28.599054 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:28.595064    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s"
Aug 05 18:04:28.599054 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:28.596894    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv" event={"ID":"0d934782-ebf8-4997-9625-0064a036b167","Type":"ContainerStarted","Data":"ff76ddb2b861016663e2a609a56118c0b1f5998b47e50970d6434b7729a36d22"}
Aug 05 18:04:28.599054 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:28.597071    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv"
Aug 05 18:04:28.615863 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:28.615827    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s" podStartSLOduration=1.615812502 podStartE2EDuration="1.615812502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:04:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:04:28.614652402 +0000 UTC m=+1875.967326996" watchObservedRunningTime="2026-08-05 18:04:28.615812502 +0000 UTC m=+1875.968487096"
Aug 05 18:04:28.637247 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:28.637208    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv" podStartSLOduration=1.6371702959999999 podStartE2EDuration="1.637170296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:04:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:04:28.636464848 +0000 UTC m=+1875.989139443" watchObservedRunningTime="2026-08-05 18:04:28.637170296 +0000 UTC m=+1875.989844890"
Aug 05 18:04:34.608094 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:34.608014    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"
Aug 05 18:04:39.604228 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:39.604173    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv"
Aug 05 18:04:39.604654 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:39.604409    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s"
Aug 05 18:04:57.666320 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:57.666288    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"]
Aug 05 18:04:57.666843 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:57.666511    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw" podUID="9c75c11c-1c32-4b96-a38d-abc2ad3e8f99" containerName="maas-api" containerID="cri-o://1011abe0e10944720f1e1f3c17a81d75f7b1ba0ab9feae131e57a7b9a50f11a8" gracePeriod=30
Aug 05 18:04:57.728971 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:57.728939    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s"]
Aug 05 18:04:57.729201 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:57.729162    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s" podUID="3ddf7bf1-7da7-4c58-a3b6-853320a7fbec" containerName="payload-processing" containerID="cri-o://cbb95d7865f6f6af952891a174e485d6aa5491c9b153c7a8b43cad88d6e35f05" gracePeriod=30
Aug 05 18:04:57.742885 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:57.742855    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv"]
Aug 05 18:04:57.743155 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:57.743091    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv" podUID="0d934782-ebf8-4997-9625-0064a036b167" containerName="payload-pre-processing" containerID="cri-o://ff76ddb2b861016663e2a609a56118c0b1f5998b47e50970d6434b7729a36d22" gracePeriod=30
Aug 05 18:04:57.843672 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:57.843646    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:57.923454 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:57.923282    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:58.015070 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.015044    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"
Aug 05 18:04:58.018638 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.018614    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv"
Aug 05 18:04:58.027753 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.027727    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s"
Aug 05 18:04:58.032852 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.032825    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c75c11c-1c32-4b96-a38d-abc2ad3e8f99-openshift-service-ca\") pod \"9c75c11c-1c32-4b96-a38d-abc2ad3e8f99\" (UID: \"9c75c11c-1c32-4b96-a38d-abc2ad3e8f99\") "
Aug 05 18:04:58.032970 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.032871    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9g7q8\" (UniqueName: \"kubernetes.io/projected/9c75c11c-1c32-4b96-a38d-abc2ad3e8f99-kube-api-access-9g7q8\") pod \"9c75c11c-1c32-4b96-a38d-abc2ad3e8f99\" (UID: \"9c75c11c-1c32-4b96-a38d-abc2ad3e8f99\") "
Aug 05 18:04:58.032970 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.032924    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlvd7\" (UniqueName: \"kubernetes.io/projected/0d934782-ebf8-4997-9625-0064a036b167-kube-api-access-wlvd7\") pod \"0d934782-ebf8-4997-9625-0064a036b167\" (UID: \"0d934782-ebf8-4997-9625-0064a036b167\") "
Aug 05 18:04:58.032970 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.032956    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d934782-ebf8-4997-9625-0064a036b167-plugins-config-volume\") pod \"0d934782-ebf8-4997-9625-0064a036b167\" (UID: \"0d934782-ebf8-4997-9625-0064a036b167\") "
Aug 05 18:04:58.033146 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.033028    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c75c11c-1c32-4b96-a38d-abc2ad3e8f99-maas-api-tls\") pod \"9c75c11c-1c32-4b96-a38d-abc2ad3e8f99\" (UID: \"9c75c11c-1c32-4b96-a38d-abc2ad3e8f99\") "
Aug 05 18:04:58.034205 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.034144    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d934782-ebf8-4997-9625-0064a036b167-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0d934782-ebf8-4997-9625-0064a036b167" (UID: "0d934782-ebf8-4997-9625-0064a036b167"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:04:58.035015 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.034980    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c75c11c-1c32-4b96-a38d-abc2ad3e8f99-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9c75c11c-1c32-4b96-a38d-abc2ad3e8f99" (UID: "9c75c11c-1c32-4b96-a38d-abc2ad3e8f99"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:04:58.035958 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.035925    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c75c11c-1c32-4b96-a38d-abc2ad3e8f99-kube-api-access-9g7q8" (OuterVolumeSpecName: "kube-api-access-9g7q8") pod "9c75c11c-1c32-4b96-a38d-abc2ad3e8f99" (UID: "9c75c11c-1c32-4b96-a38d-abc2ad3e8f99"). InnerVolumeSpecName "kube-api-access-9g7q8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:04:58.036048 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.035956    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c75c11c-1c32-4b96-a38d-abc2ad3e8f99-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9c75c11c-1c32-4b96-a38d-abc2ad3e8f99" (UID: "9c75c11c-1c32-4b96-a38d-abc2ad3e8f99"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:04:58.037742 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.037707    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d934782-ebf8-4997-9625-0064a036b167-kube-api-access-wlvd7" (OuterVolumeSpecName: "kube-api-access-wlvd7") pod "0d934782-ebf8-4997-9625-0064a036b167" (UID: "0d934782-ebf8-4997-9625-0064a036b167"). InnerVolumeSpecName "kube-api-access-wlvd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:04:58.109775 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.109740    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:58.133837 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.133801    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnzz5\" (UniqueName: \"kubernetes.io/projected/3ddf7bf1-7da7-4c58-a3b6-853320a7fbec-kube-api-access-qnzz5\") pod \"3ddf7bf1-7da7-4c58-a3b6-853320a7fbec\" (UID: \"3ddf7bf1-7da7-4c58-a3b6-853320a7fbec\") "
Aug 05 18:04:58.133999 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.133931    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ddf7bf1-7da7-4c58-a3b6-853320a7fbec-plugins-config-volume\") pod \"3ddf7bf1-7da7-4c58-a3b6-853320a7fbec\" (UID: \"3ddf7bf1-7da7-4c58-a3b6-853320a7fbec\") "
Aug 05 18:04:58.134162 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.134148    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c75c11c-1c32-4b96-a38d-abc2ad3e8f99-openshift-service-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:04:58.134235 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.134166    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9g7q8\" (UniqueName: \"kubernetes.io/projected/9c75c11c-1c32-4b96-a38d-abc2ad3e8f99-kube-api-access-9g7q8\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:04:58.134235 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.134193    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlvd7\" (UniqueName: \"kubernetes.io/projected/0d934782-ebf8-4997-9625-0064a036b167-kube-api-access-wlvd7\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:04:58.134235 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.134203    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d934782-ebf8-4997-9625-0064a036b167-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:04:58.134235 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.134214    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c75c11c-1c32-4b96-a38d-abc2ad3e8f99-maas-api-tls\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:04:58.134371 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.134281    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ddf7bf1-7da7-4c58-a3b6-853320a7fbec-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3ddf7bf1-7da7-4c58-a3b6-853320a7fbec" (UID: "3ddf7bf1-7da7-4c58-a3b6-853320a7fbec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:04:58.136161 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.136132    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ddf7bf1-7da7-4c58-a3b6-853320a7fbec-kube-api-access-qnzz5" (OuterVolumeSpecName: "kube-api-access-qnzz5") pod "3ddf7bf1-7da7-4c58-a3b6-853320a7fbec" (UID: "3ddf7bf1-7da7-4c58-a3b6-853320a7fbec"). InnerVolumeSpecName "kube-api-access-qnzz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:04:58.234775 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.234674    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ddf7bf1-7da7-4c58-a3b6-853320a7fbec-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:04:58.234775 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.234705    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnzz5\" (UniqueName: \"kubernetes.io/projected/3ddf7bf1-7da7-4c58-a3b6-853320a7fbec-kube-api-access-qnzz5\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:04:58.314171 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.314129    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:58.541110 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.541031    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:58.762131 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.762100    2569 generic.go:356] "Generic (PLEG): container finished" podID="3ddf7bf1-7da7-4c58-a3b6-853320a7fbec" containerID="cbb95d7865f6f6af952891a174e485d6aa5491c9b153c7a8b43cad88d6e35f05" exitCode=0
Aug 05 18:04:58.762649 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.762154    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s"
Aug 05 18:04:58.762649 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.762212    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s" event={"ID":"3ddf7bf1-7da7-4c58-a3b6-853320a7fbec","Type":"ContainerDied","Data":"cbb95d7865f6f6af952891a174e485d6aa5491c9b153c7a8b43cad88d6e35f05"}
Aug 05 18:04:58.762649 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.762273    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s" event={"ID":"3ddf7bf1-7da7-4c58-a3b6-853320a7fbec","Type":"ContainerDied","Data":"a8afb2ccf05758edb607e71500f9156a791897229d5460898a64c31e581a6a33"}
Aug 05 18:04:58.762649 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.762304    2569 scope.go:117] "RemoveContainer" containerID="cbb95d7865f6f6af952891a174e485d6aa5491c9b153c7a8b43cad88d6e35f05"
Aug 05 18:04:58.763634 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.763511    2569 generic.go:356] "Generic (PLEG): container finished" podID="9c75c11c-1c32-4b96-a38d-abc2ad3e8f99" containerID="1011abe0e10944720f1e1f3c17a81d75f7b1ba0ab9feae131e57a7b9a50f11a8" exitCode=0
Aug 05 18:04:58.763634 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.763586    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"
Aug 05 18:04:58.763797 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.763584    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw" event={"ID":"9c75c11c-1c32-4b96-a38d-abc2ad3e8f99","Type":"ContainerDied","Data":"1011abe0e10944720f1e1f3c17a81d75f7b1ba0ab9feae131e57a7b9a50f11a8"}
Aug 05 18:04:58.763797 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.763717    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw" event={"ID":"9c75c11c-1c32-4b96-a38d-abc2ad3e8f99","Type":"ContainerDied","Data":"8752f28b625c1ffefdc7d69d1e65bc4ff4f09bd34ad74a4ff7f7965bb4d68250"}
Aug 05 18:04:58.764901 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.764878    2569 generic.go:356] "Generic (PLEG): container finished" podID="0d934782-ebf8-4997-9625-0064a036b167" containerID="ff76ddb2b861016663e2a609a56118c0b1f5998b47e50970d6434b7729a36d22" exitCode=0
Aug 05 18:04:58.765046 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.764912    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv"
Aug 05 18:04:58.765046 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.764939    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv" event={"ID":"0d934782-ebf8-4997-9625-0064a036b167","Type":"ContainerDied","Data":"ff76ddb2b861016663e2a609a56118c0b1f5998b47e50970d6434b7729a36d22"}
Aug 05 18:04:58.765046 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.764988    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv" event={"ID":"0d934782-ebf8-4997-9625-0064a036b167","Type":"ContainerDied","Data":"d66317f88264af3d5f17bd1fcba72d626553d123dc6d4158597fc5f29268e8e5"}
Aug 05 18:04:58.771596 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.771575    2569 scope.go:117] "RemoveContainer" containerID="cbb95d7865f6f6af952891a174e485d6aa5491c9b153c7a8b43cad88d6e35f05"
Aug 05 18:04:58.771858 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:04:58.771836    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbb95d7865f6f6af952891a174e485d6aa5491c9b153c7a8b43cad88d6e35f05\": container with ID starting with cbb95d7865f6f6af952891a174e485d6aa5491c9b153c7a8b43cad88d6e35f05 not found: ID does not exist" containerID="cbb95d7865f6f6af952891a174e485d6aa5491c9b153c7a8b43cad88d6e35f05"
Aug 05 18:04:58.771947 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.771870    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbb95d7865f6f6af952891a174e485d6aa5491c9b153c7a8b43cad88d6e35f05"} err="failed to get container status \"cbb95d7865f6f6af952891a174e485d6aa5491c9b153c7a8b43cad88d6e35f05\": rpc error: code = NotFound desc = could not find container \"cbb95d7865f6f6af952891a174e485d6aa5491c9b153c7a8b43cad88d6e35f05\": container with ID starting with cbb95d7865f6f6af952891a174e485d6aa5491c9b153c7a8b43cad88d6e35f05 not found: ID does not exist"
Aug 05 18:04:58.771947 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.771892    2569 scope.go:117] "RemoveContainer" containerID="1011abe0e10944720f1e1f3c17a81d75f7b1ba0ab9feae131e57a7b9a50f11a8"
Aug 05 18:04:58.781351 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.781335    2569 scope.go:117] "RemoveContainer" containerID="1011abe0e10944720f1e1f3c17a81d75f7b1ba0ab9feae131e57a7b9a50f11a8"
Aug 05 18:04:58.781615 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:04:58.781598    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1011abe0e10944720f1e1f3c17a81d75f7b1ba0ab9feae131e57a7b9a50f11a8\": container with ID starting with 1011abe0e10944720f1e1f3c17a81d75f7b1ba0ab9feae131e57a7b9a50f11a8 not found: ID does not exist" containerID="1011abe0e10944720f1e1f3c17a81d75f7b1ba0ab9feae131e57a7b9a50f11a8"
Aug 05 18:04:58.781675 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.781621    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1011abe0e10944720f1e1f3c17a81d75f7b1ba0ab9feae131e57a7b9a50f11a8"} err="failed to get container status \"1011abe0e10944720f1e1f3c17a81d75f7b1ba0ab9feae131e57a7b9a50f11a8\": rpc error: code = NotFound desc = could not find container \"1011abe0e10944720f1e1f3c17a81d75f7b1ba0ab9feae131e57a7b9a50f11a8\": container with ID starting with 1011abe0e10944720f1e1f3c17a81d75f7b1ba0ab9feae131e57a7b9a50f11a8 not found: ID does not exist"
Aug 05 18:04:58.781675 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.781637    2569 scope.go:117] "RemoveContainer" containerID="ff76ddb2b861016663e2a609a56118c0b1f5998b47e50970d6434b7729a36d22"
Aug 05 18:04:58.789540 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.789522    2569 scope.go:117] "RemoveContainer" containerID="ff76ddb2b861016663e2a609a56118c0b1f5998b47e50970d6434b7729a36d22"
Aug 05 18:04:58.789820 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:04:58.789802    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff76ddb2b861016663e2a609a56118c0b1f5998b47e50970d6434b7729a36d22\": container with ID starting with ff76ddb2b861016663e2a609a56118c0b1f5998b47e50970d6434b7729a36d22 not found: ID does not exist" containerID="ff76ddb2b861016663e2a609a56118c0b1f5998b47e50970d6434b7729a36d22"
Aug 05 18:04:58.789893 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.789829    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff76ddb2b861016663e2a609a56118c0b1f5998b47e50970d6434b7729a36d22"} err="failed to get container status \"ff76ddb2b861016663e2a609a56118c0b1f5998b47e50970d6434b7729a36d22\": rpc error: code = NotFound desc = could not find container \"ff76ddb2b861016663e2a609a56118c0b1f5998b47e50970d6434b7729a36d22\": container with ID starting with ff76ddb2b861016663e2a609a56118c0b1f5998b47e50970d6434b7729a36d22 not found: ID does not exist"
Aug 05 18:04:58.798884 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.798838    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s"]
Aug 05 18:04:58.812568 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.812549    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e7e381dc-849d4cbd77-jv45s"]
Aug 05 18:04:58.835511 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.835482    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv"]
Aug 05 18:04:58.837685 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.837662    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e7e381dc-5f4f949d9b-chlnv"]
Aug 05 18:04:58.851574 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.851550    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"]
Aug 05 18:04:58.854937 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.854917    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e7e381dc-d75677f68-5nvxw"]
Aug 05 18:04:58.874813 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:58.874785    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:59.091269 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:59.091241    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:59.151993 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:59.151956    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d934782-ebf8-4997-9625-0064a036b167" path="/var/lib/kubelet/pods/0d934782-ebf8-4997-9625-0064a036b167/volumes"
Aug 05 18:04:59.152409 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:59.152389    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ddf7bf1-7da7-4c58-a3b6-853320a7fbec" path="/var/lib/kubelet/pods/3ddf7bf1-7da7-4c58-a3b6-853320a7fbec/volumes"
Aug 05 18:04:59.152796 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:04:59.152778    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c75c11c-1c32-4b96-a38d-abc2ad3e8f99" path="/var/lib/kubelet/pods/9c75c11c-1c32-4b96-a38d-abc2ad3e8f99/volumes"
Aug 05 18:05:01.113836 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:01.113786    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:01.339144 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:01.339110    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:02.315812 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.315781    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"]
Aug 05 18:05:02.316259 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.316246    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c75c11c-1c32-4b96-a38d-abc2ad3e8f99" containerName="maas-api"
Aug 05 18:05:02.316319 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.316261    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c75c11c-1c32-4b96-a38d-abc2ad3e8f99" containerName="maas-api"
Aug 05 18:05:02.316319 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.316274    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d934782-ebf8-4997-9625-0064a036b167" containerName="payload-pre-processing"
Aug 05 18:05:02.316319 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.316280    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d934782-ebf8-4997-9625-0064a036b167" containerName="payload-pre-processing"
Aug 05 18:05:02.316319 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.316300    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ddf7bf1-7da7-4c58-a3b6-853320a7fbec" containerName="payload-processing"
Aug 05 18:05:02.316319 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.316309    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ddf7bf1-7da7-4c58-a3b6-853320a7fbec" containerName="payload-processing"
Aug 05 18:05:02.316467 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.316383    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c75c11c-1c32-4b96-a38d-abc2ad3e8f99" containerName="maas-api"
Aug 05 18:05:02.316467 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.316393    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d934782-ebf8-4997-9625-0064a036b167" containerName="payload-pre-processing"
Aug 05 18:05:02.316467 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.316402    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ddf7bf1-7da7-4c58-a3b6-853320a7fbec" containerName="payload-processing"
Aug 05 18:05:02.321265 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.321247    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"
Aug 05 18:05:02.325328 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.325301    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jdhhr\""
Aug 05 18:05:02.325446 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.325304    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-efc0f562-serving-cert\""
Aug 05 18:05:02.335535 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.335512    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"]
Aug 05 18:05:02.362358 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.362328    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:02.372580 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.372550    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-maas-api-tls\") pod \"maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2\" (UID: \"5d9155c5-dbfd-468a-b1df-9b76995b1cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"
Aug 05 18:05:02.372713 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.372636    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgxkr\" (UniqueName: \"kubernetes.io/projected/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-kube-api-access-qgxkr\") pod \"maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2\" (UID: \"5d9155c5-dbfd-468a-b1df-9b76995b1cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"
Aug 05 18:05:02.372713 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.372685    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-openshift-service-ca\") pod \"maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2\" (UID: \"5d9155c5-dbfd-468a-b1df-9b76995b1cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"
Aug 05 18:05:02.473419 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.473391    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-openshift-service-ca\") pod \"maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2\" (UID: \"5d9155c5-dbfd-468a-b1df-9b76995b1cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"
Aug 05 18:05:02.473539 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.473443    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-maas-api-tls\") pod \"maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2\" (UID: \"5d9155c5-dbfd-468a-b1df-9b76995b1cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"
Aug 05 18:05:02.473539 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.473512    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgxkr\" (UniqueName: \"kubernetes.io/projected/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-kube-api-access-qgxkr\") pod \"maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2\" (UID: \"5d9155c5-dbfd-468a-b1df-9b76995b1cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"
Aug 05 18:05:02.473618 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:05:02.473607    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-serving-cert: secret "maas-api-e2e-ait-efc0f562-serving-cert" not found
Aug 05 18:05:02.473674 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:05:02.473664    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-maas-api-tls podName:5d9155c5-dbfd-468a-b1df-9b76995b1cfd nodeName:}" failed. No retries permitted until 2026-08-05 18:05:02.973649134 +0000 UTC m=+1910.326323706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-maas-api-tls") pod "maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2" (UID: "5d9155c5-dbfd-468a-b1df-9b76995b1cfd") : secret "maas-api-e2e-ait-efc0f562-serving-cert" not found
Aug 05 18:05:02.474093 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.474077    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-openshift-service-ca\") pod \"maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2\" (UID: \"5d9155c5-dbfd-468a-b1df-9b76995b1cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"
Aug 05 18:05:02.484255 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.484230    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgxkr\" (UniqueName: \"kubernetes.io/projected/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-kube-api-access-qgxkr\") pod \"maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2\" (UID: \"5d9155c5-dbfd-468a-b1df-9b76995b1cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"
Aug 05 18:05:02.978667 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:02.978633    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-maas-api-tls\") pod \"maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2\" (UID: \"5d9155c5-dbfd-468a-b1df-9b76995b1cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"
Aug 05 18:05:02.978841 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:05:02.978748    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-serving-cert: secret "maas-api-e2e-ait-efc0f562-serving-cert" not found
Aug 05 18:05:02.978841 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:05:02.978808    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-maas-api-tls podName:5d9155c5-dbfd-468a-b1df-9b76995b1cfd nodeName:}" failed. No retries permitted until 2026-08-05 18:05:03.978793914 +0000 UTC m=+1911.331468485 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-maas-api-tls") pod "maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2" (UID: "5d9155c5-dbfd-468a-b1df-9b76995b1cfd") : secret "maas-api-e2e-ait-efc0f562-serving-cert" not found
Aug 05 18:05:03.049032 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.048973    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:03.194212 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.194160    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:03.270413 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.270342    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv"]
Aug 05 18:05:03.274112 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.274097    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv"
Aug 05 18:05:03.278113 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.278085    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-efc0f562\""
Aug 05 18:05:03.313073 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.313049    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv"]
Aug 05 18:05:03.380961 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.380935    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8347a728-4b63-461a-a3d9-67d6ca2a91ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv\" (UID: \"8347a728-4b63-461a-a3d9-67d6ca2a91ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv"
Aug 05 18:05:03.381358 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.381102    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c4gh\" (UniqueName: \"kubernetes.io/projected/8347a728-4b63-461a-a3d9-67d6ca2a91ee-kube-api-access-7c4gh\") pod \"payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv\" (UID: \"8347a728-4b63-461a-a3d9-67d6ca2a91ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv"
Aug 05 18:05:03.388094 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.388071    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7"]
Aug 05 18:05:03.391622 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.391599    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7"
Aug 05 18:05:03.394909 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.394888    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-efc0f562-dockercfg-mctg9\""
Aug 05 18:05:03.407279 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.407258    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7"]
Aug 05 18:05:03.432720 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.432691    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:03.482601 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.482571    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c4gh\" (UniqueName: \"kubernetes.io/projected/8347a728-4b63-461a-a3d9-67d6ca2a91ee-kube-api-access-7c4gh\") pod \"payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv\" (UID: \"8347a728-4b63-461a-a3d9-67d6ca2a91ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv"
Aug 05 18:05:03.482722 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.482610    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c-plugins-config-volume\") pod \"payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7\" (UID: \"b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c\") " pod="openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7"
Aug 05 18:05:03.482722 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.482635    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx287\" (UniqueName: \"kubernetes.io/projected/b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c-kube-api-access-lx287\") pod \"payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7\" (UID: \"b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c\") " pod="openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7"
Aug 05 18:05:03.482722 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.482713    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8347a728-4b63-461a-a3d9-67d6ca2a91ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv\" (UID: \"8347a728-4b63-461a-a3d9-67d6ca2a91ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv"
Aug 05 18:05:03.483519 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.483495    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8347a728-4b63-461a-a3d9-67d6ca2a91ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv\" (UID: \"8347a728-4b63-461a-a3d9-67d6ca2a91ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv"
Aug 05 18:05:03.493160 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.493140    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c4gh\" (UniqueName: \"kubernetes.io/projected/8347a728-4b63-461a-a3d9-67d6ca2a91ee-kube-api-access-7c4gh\") pod \"payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv\" (UID: \"8347a728-4b63-461a-a3d9-67d6ca2a91ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv"
Aug 05 18:05:03.583288 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.583262    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c-plugins-config-volume\") pod \"payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7\" (UID: \"b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c\") " pod="openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7"
Aug 05 18:05:03.583288 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.583299    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx287\" (UniqueName: \"kubernetes.io/projected/b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c-kube-api-access-lx287\") pod \"payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7\" (UID: \"b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c\") " pod="openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7"
Aug 05 18:05:03.583962 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.583936    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c-plugins-config-volume\") pod \"payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7\" (UID: \"b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c\") " pod="openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7"
Aug 05 18:05:03.584529 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.584509    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv"
Aug 05 18:05:03.597098 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.596217    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx287\" (UniqueName: \"kubernetes.io/projected/b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c-kube-api-access-lx287\") pod \"payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7\" (UID: \"b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c\") " pod="openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7"
Aug 05 18:05:03.702463 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.702435    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7"
Aug 05 18:05:03.732264 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.732237    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv"]
Aug 05 18:05:03.734365 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:05:03.734339    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8347a728_4b63_461a_a3d9_67d6ca2a91ee.slice/crio-da5bdd0538e993c010dfbef6f9acc2acb8dc42fa39d67846b11f8fe05524435e WatchSource:0}: Error finding container da5bdd0538e993c010dfbef6f9acc2acb8dc42fa39d67846b11f8fe05524435e: Status 404 returned error can't find the container with id da5bdd0538e993c010dfbef6f9acc2acb8dc42fa39d67846b11f8fe05524435e
Aug 05 18:05:03.796103 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.796073    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv" event={"ID":"8347a728-4b63-461a-a3d9-67d6ca2a91ee","Type":"ContainerStarted","Data":"da5bdd0538e993c010dfbef6f9acc2acb8dc42fa39d67846b11f8fe05524435e"}
Aug 05 18:05:03.867040 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.867000    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7"]
Aug 05 18:05:03.871382 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:05:03.871355    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4747837_a5b0_4a32_a3ba_d3fdd9a3c12c.slice/crio-c81a8871f70b1ee567ce1e3cdd92e94e0cdd37cf812f4ce5d2c194cadb82c92e WatchSource:0}: Error finding container c81a8871f70b1ee567ce1e3cdd92e94e0cdd37cf812f4ce5d2c194cadb82c92e: Status 404 returned error can't find the container with id c81a8871f70b1ee567ce1e3cdd92e94e0cdd37cf812f4ce5d2c194cadb82c92e
Aug 05 18:05:03.988025 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.987996    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-maas-api-tls\") pod \"maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2\" (UID: \"5d9155c5-dbfd-468a-b1df-9b76995b1cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"
Aug 05 18:05:03.990548 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:03.990525    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-maas-api-tls\") pod \"maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2\" (UID: \"5d9155c5-dbfd-468a-b1df-9b76995b1cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"
Aug 05 18:05:04.132527 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:04.132453    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"
Aug 05 18:05:04.289265 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:04.289242    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"]
Aug 05 18:05:04.292039 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:05:04.292010    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d9155c5_dbfd_468a_b1df_9b76995b1cfd.slice/crio-6800b19ac24c2254629d041320922d5030f08c4374a06a71c531bdc413265c8e WatchSource:0}: Error finding container 6800b19ac24c2254629d041320922d5030f08c4374a06a71c531bdc413265c8e: Status 404 returned error can't find the container with id 6800b19ac24c2254629d041320922d5030f08c4374a06a71c531bdc413265c8e
Aug 05 18:05:04.802941 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:04.802910    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv" event={"ID":"8347a728-4b63-461a-a3d9-67d6ca2a91ee","Type":"ContainerStarted","Data":"36071019a4cb5d814ee78c5593ae58162c3d0ec2beefbe3c3593157210693868"}
Aug 05 18:05:04.803264 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:04.803046    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv"
Aug 05 18:05:04.804135 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:04.804114    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2" event={"ID":"5d9155c5-dbfd-468a-b1df-9b76995b1cfd","Type":"ContainerStarted","Data":"6800b19ac24c2254629d041320922d5030f08c4374a06a71c531bdc413265c8e"}
Aug 05 18:05:04.805474 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:04.805455    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7" event={"ID":"b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c","Type":"ContainerStarted","Data":"fbbbf03a83d7937345adb255cfbbf2c762cc024fa1cf7aadc5d99a9e2dcb2571"}
Aug 05 18:05:04.805564 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:04.805492    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7" event={"ID":"b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c","Type":"ContainerStarted","Data":"c81a8871f70b1ee567ce1e3cdd92e94e0cdd37cf812f4ce5d2c194cadb82c92e"}
Aug 05 18:05:04.805564 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:04.805513    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7"
Aug 05 18:05:04.826324 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:04.826281    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv" podStartSLOduration=1.826268787 podStartE2EDuration="1.826268787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:05:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:05:04.824077217 +0000 UTC m=+1912.176751812" watchObservedRunningTime="2026-08-05 18:05:04.826268787 +0000 UTC m=+1912.178943446"
Aug 05 18:05:04.847057 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:04.847010    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7" podStartSLOduration=1.846993233 podStartE2EDuration="1.846993233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:05:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:05:04.845094016 +0000 UTC m=+1912.197768611" watchObservedRunningTime="2026-08-05 18:05:04.846993233 +0000 UTC m=+1912.199667836"
Aug 05 18:05:05.812399 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:05.812366    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2" event={"ID":"5d9155c5-dbfd-468a-b1df-9b76995b1cfd","Type":"ContainerStarted","Data":"a963511e205fd988209e88cd7e36f3b1e6ef468641a6c67c9f4cd8075cdc0cf9"}
Aug 05 18:05:05.812805 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:05.812550    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"
Aug 05 18:05:05.836550 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:05.836506    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2" podStartSLOduration=3.362180944 podStartE2EDuration="3.836490792s" totalImagesPullingTime="474.309848ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:05:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:05:05.835101165 +0000 UTC m=+1913.187775759" watchObservedRunningTime="2026-08-05 18:05:05.836490792 +0000 UTC m=+1913.189165386"
Aug 05 18:05:11.822913 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:11.822884    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"
Aug 05 18:05:15.814328 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:15.814281    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv" podUID="8347a728-4b63-461a-a3d9-67d6ca2a91ee" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.53:9004: connect: connection refused"
Aug 05 18:05:15.814767 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:15.814744    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7"
Aug 05 18:05:25.812949 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:25.812917    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv"
Aug 05 18:05:42.889291 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:42.889260    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"]
Aug 05 18:05:42.889859 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:42.889479    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2" podUID="5d9155c5-dbfd-468a-b1df-9b76995b1cfd" containerName="maas-api" containerID="cri-o://a963511e205fd988209e88cd7e36f3b1e6ef468641a6c67c9f4cd8075cdc0cf9" gracePeriod=30
Aug 05 18:05:42.932351 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:42.932325    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:42.946230 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:42.946199    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7"]
Aug 05 18:05:42.946495 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:42.946468    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7" podUID="b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c" containerName="payload-processing" containerID="cri-o://fbbbf03a83d7937345adb255cfbbf2c762cc024fa1cf7aadc5d99a9e2dcb2571" gracePeriod=30
Aug 05 18:05:42.955328 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:42.955299    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv"]
Aug 05 18:05:42.955564 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:42.955511    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv" podUID="8347a728-4b63-461a-a3d9-67d6ca2a91ee" containerName="payload-pre-processing" containerID="cri-o://36071019a4cb5d814ee78c5593ae58162c3d0ec2beefbe3c3593157210693868" gracePeriod=30
Aug 05 18:05:43.005892 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.005862    2569 generic.go:356] "Generic (PLEG): container finished" podID="5d9155c5-dbfd-468a-b1df-9b76995b1cfd" containerID="a963511e205fd988209e88cd7e36f3b1e6ef468641a6c67c9f4cd8075cdc0cf9" exitCode=0
Aug 05 18:05:43.006049 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.005932    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2" event={"ID":"5d9155c5-dbfd-468a-b1df-9b76995b1cfd","Type":"ContainerDied","Data":"a963511e205fd988209e88cd7e36f3b1e6ef468641a6c67c9f4cd8075cdc0cf9"}
Aug 05 18:05:43.035445 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.035414    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:43.111908 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.111864    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:43.233581 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.233285    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7"
Aug 05 18:05:43.275807 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.275776    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"
Aug 05 18:05:43.280020 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.279996    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv"
Aug 05 18:05:43.336379 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.336347    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c-plugins-config-volume\") pod \"b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c\" (UID: \"b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c\") "
Aug 05 18:05:43.336379 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.336382    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx287\" (UniqueName: \"kubernetes.io/projected/b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c-kube-api-access-lx287\") pod \"b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c\" (UID: \"b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c\") "
Aug 05 18:05:43.336756 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.336727    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c" (UID: "b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:05:43.338638 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.338611    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c-kube-api-access-lx287" (OuterVolumeSpecName: "kube-api-access-lx287") pod "b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c" (UID: "b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c"). InnerVolumeSpecName "kube-api-access-lx287". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:05:43.437386 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.437315    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgxkr\" (UniqueName: \"kubernetes.io/projected/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-kube-api-access-qgxkr\") pod \"5d9155c5-dbfd-468a-b1df-9b76995b1cfd\" (UID: \"5d9155c5-dbfd-468a-b1df-9b76995b1cfd\") "
Aug 05 18:05:43.437386 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.437354    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-maas-api-tls\") pod \"5d9155c5-dbfd-468a-b1df-9b76995b1cfd\" (UID: \"5d9155c5-dbfd-468a-b1df-9b76995b1cfd\") "
Aug 05 18:05:43.437570 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.437419    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-openshift-service-ca\") pod \"5d9155c5-dbfd-468a-b1df-9b76995b1cfd\" (UID: \"5d9155c5-dbfd-468a-b1df-9b76995b1cfd\") "
Aug 05 18:05:43.437570 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.437442    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8347a728-4b63-461a-a3d9-67d6ca2a91ee-plugins-config-volume\") pod \"8347a728-4b63-461a-a3d9-67d6ca2a91ee\" (UID: \"8347a728-4b63-461a-a3d9-67d6ca2a91ee\") "
Aug 05 18:05:43.437570 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.437472    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7c4gh\" (UniqueName: \"kubernetes.io/projected/8347a728-4b63-461a-a3d9-67d6ca2a91ee-kube-api-access-7c4gh\") pod \"8347a728-4b63-461a-a3d9-67d6ca2a91ee\" (UID: \"8347a728-4b63-461a-a3d9-67d6ca2a91ee\") "
Aug 05 18:05:43.437737 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.437689    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:05:43.437737 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.437708    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx287\" (UniqueName: \"kubernetes.io/projected/b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c-kube-api-access-lx287\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:05:43.437828 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.437766    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5d9155c5-dbfd-468a-b1df-9b76995b1cfd" (UID: "5d9155c5-dbfd-468a-b1df-9b76995b1cfd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:05:43.437874 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.437823    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8347a728-4b63-461a-a3d9-67d6ca2a91ee-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8347a728-4b63-461a-a3d9-67d6ca2a91ee" (UID: "8347a728-4b63-461a-a3d9-67d6ca2a91ee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:05:43.439560 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.439542    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8347a728-4b63-461a-a3d9-67d6ca2a91ee-kube-api-access-7c4gh" (OuterVolumeSpecName: "kube-api-access-7c4gh") pod "8347a728-4b63-461a-a3d9-67d6ca2a91ee" (UID: "8347a728-4b63-461a-a3d9-67d6ca2a91ee"). InnerVolumeSpecName "kube-api-access-7c4gh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:05:43.439935 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.439919    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5d9155c5-dbfd-468a-b1df-9b76995b1cfd" (UID: "5d9155c5-dbfd-468a-b1df-9b76995b1cfd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:05:43.439983 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.439938    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-kube-api-access-qgxkr" (OuterVolumeSpecName: "kube-api-access-qgxkr") pod "5d9155c5-dbfd-468a-b1df-9b76995b1cfd" (UID: "5d9155c5-dbfd-468a-b1df-9b76995b1cfd"). InnerVolumeSpecName "kube-api-access-qgxkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:05:43.538460 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.538428    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-maas-api-tls\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:05:43.538460 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.538454    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-openshift-service-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:05:43.538460 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.538463    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8347a728-4b63-461a-a3d9-67d6ca2a91ee-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:05:43.538681 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.538473    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7c4gh\" (UniqueName: \"kubernetes.io/projected/8347a728-4b63-461a-a3d9-67d6ca2a91ee-kube-api-access-7c4gh\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:05:43.538681 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:43.538483    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgxkr\" (UniqueName: \"kubernetes.io/projected/5d9155c5-dbfd-468a-b1df-9b76995b1cfd-kube-api-access-qgxkr\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:05:44.012514 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.012481    2569 generic.go:356] "Generic (PLEG): container finished" podID="8347a728-4b63-461a-a3d9-67d6ca2a91ee" containerID="36071019a4cb5d814ee78c5593ae58162c3d0ec2beefbe3c3593157210693868" exitCode=0
Aug 05 18:05:44.012955 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.012529    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv"
Aug 05 18:05:44.012955 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.012558    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv" event={"ID":"8347a728-4b63-461a-a3d9-67d6ca2a91ee","Type":"ContainerDied","Data":"36071019a4cb5d814ee78c5593ae58162c3d0ec2beefbe3c3593157210693868"}
Aug 05 18:05:44.012955 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.012610    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv" event={"ID":"8347a728-4b63-461a-a3d9-67d6ca2a91ee","Type":"ContainerDied","Data":"da5bdd0538e993c010dfbef6f9acc2acb8dc42fa39d67846b11f8fe05524435e"}
Aug 05 18:05:44.012955 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.012642    2569 scope.go:117] "RemoveContainer" containerID="36071019a4cb5d814ee78c5593ae58162c3d0ec2beefbe3c3593157210693868"
Aug 05 18:05:44.014033 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.014001    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"
Aug 05 18:05:44.014033 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.014018    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2" event={"ID":"5d9155c5-dbfd-468a-b1df-9b76995b1cfd","Type":"ContainerDied","Data":"6800b19ac24c2254629d041320922d5030f08c4374a06a71c531bdc413265c8e"}
Aug 05 18:05:44.015211 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.015165    2569 generic.go:356] "Generic (PLEG): container finished" podID="b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c" containerID="fbbbf03a83d7937345adb255cfbbf2c762cc024fa1cf7aadc5d99a9e2dcb2571" exitCode=0
Aug 05 18:05:44.015335 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.015320    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7" event={"ID":"b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c","Type":"ContainerDied","Data":"fbbbf03a83d7937345adb255cfbbf2c762cc024fa1cf7aadc5d99a9e2dcb2571"}
Aug 05 18:05:44.015411 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.015366    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7" event={"ID":"b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c","Type":"ContainerDied","Data":"c81a8871f70b1ee567ce1e3cdd92e94e0cdd37cf812f4ce5d2c194cadb82c92e"}
Aug 05 18:05:44.015472 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.015431    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7"
Aug 05 18:05:44.023577 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.023555    2569 scope.go:117] "RemoveContainer" containerID="36071019a4cb5d814ee78c5593ae58162c3d0ec2beefbe3c3593157210693868"
Aug 05 18:05:44.023823 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:05:44.023799    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36071019a4cb5d814ee78c5593ae58162c3d0ec2beefbe3c3593157210693868\": container with ID starting with 36071019a4cb5d814ee78c5593ae58162c3d0ec2beefbe3c3593157210693868 not found: ID does not exist" containerID="36071019a4cb5d814ee78c5593ae58162c3d0ec2beefbe3c3593157210693868"
Aug 05 18:05:44.023953 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.023838    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36071019a4cb5d814ee78c5593ae58162c3d0ec2beefbe3c3593157210693868"} err="failed to get container status \"36071019a4cb5d814ee78c5593ae58162c3d0ec2beefbe3c3593157210693868\": rpc error: code = NotFound desc = could not find container \"36071019a4cb5d814ee78c5593ae58162c3d0ec2beefbe3c3593157210693868\": container with ID starting with 36071019a4cb5d814ee78c5593ae58162c3d0ec2beefbe3c3593157210693868 not found: ID does not exist"
Aug 05 18:05:44.023953 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.023866    2569 scope.go:117] "RemoveContainer" containerID="a963511e205fd988209e88cd7e36f3b1e6ef468641a6c67c9f4cd8075cdc0cf9"
Aug 05 18:05:44.032449 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.032433    2569 scope.go:117] "RemoveContainer" containerID="fbbbf03a83d7937345adb255cfbbf2c762cc024fa1cf7aadc5d99a9e2dcb2571"
Aug 05 18:05:44.040389 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.040364    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv"]
Aug 05 18:05:44.042000 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.041978    2569 scope.go:117] "RemoveContainer" containerID="fbbbf03a83d7937345adb255cfbbf2c762cc024fa1cf7aadc5d99a9e2dcb2571"
Aug 05 18:05:44.042363 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:05:44.042341    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbbbf03a83d7937345adb255cfbbf2c762cc024fa1cf7aadc5d99a9e2dcb2571\": container with ID starting with fbbbf03a83d7937345adb255cfbbf2c762cc024fa1cf7aadc5d99a9e2dcb2571 not found: ID does not exist" containerID="fbbbf03a83d7937345adb255cfbbf2c762cc024fa1cf7aadc5d99a9e2dcb2571"
Aug 05 18:05:44.042468 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.042373    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbbbf03a83d7937345adb255cfbbf2c762cc024fa1cf7aadc5d99a9e2dcb2571"} err="failed to get container status \"fbbbf03a83d7937345adb255cfbbf2c762cc024fa1cf7aadc5d99a9e2dcb2571\": rpc error: code = NotFound desc = could not find container \"fbbbf03a83d7937345adb255cfbbf2c762cc024fa1cf7aadc5d99a9e2dcb2571\": container with ID starting with fbbbf03a83d7937345adb255cfbbf2c762cc024fa1cf7aadc5d99a9e2dcb2571 not found: ID does not exist"
Aug 05 18:05:44.042856 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.042836    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-efc0f562-68d8d7dc5b-bh4kv"]
Aug 05 18:05:44.053846 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.053828    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7"]
Aug 05 18:05:44.057648 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.057621    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-efc0f562-658795d7f4-d8mg7"]
Aug 05 18:05:44.075090 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.075070    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"]
Aug 05 18:05:44.080284 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.080263    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-efc0f562-5fb5f6f946-447d2"]
Aug 05 18:05:44.277916 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.277839    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:44.492350 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.492315    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:44.721344 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:44.721307    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:45.096718 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:45.096667    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:45.151084 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:45.151048    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d9155c5-dbfd-468a-b1df-9b76995b1cfd" path="/var/lib/kubelet/pods/5d9155c5-dbfd-468a-b1df-9b76995b1cfd/volumes"
Aug 05 18:05:45.151489 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:45.151472    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8347a728-4b63-461a-a3d9-67d6ca2a91ee" path="/var/lib/kubelet/pods/8347a728-4b63-461a-a3d9-67d6ca2a91ee/volumes"
Aug 05 18:05:45.151811 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:45.151798    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c" path="/var/lib/kubelet/pods/b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c/volumes"
Aug 05 18:05:45.341451 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:45.341422    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:47.257570 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:47.257515    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:47.357850 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:47.357820    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:48.508276 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.508243    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"]
Aug 05 18:05:48.508745 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.508641    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c" containerName="payload-processing"
Aug 05 18:05:48.508745 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.508652    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c" containerName="payload-processing"
Aug 05 18:05:48.508745 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.508660    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d9155c5-dbfd-468a-b1df-9b76995b1cfd" containerName="maas-api"
Aug 05 18:05:48.508745 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.508666    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d9155c5-dbfd-468a-b1df-9b76995b1cfd" containerName="maas-api"
Aug 05 18:05:48.508745 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.508678    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8347a728-4b63-461a-a3d9-67d6ca2a91ee" containerName="payload-pre-processing"
Aug 05 18:05:48.508745 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.508684    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8347a728-4b63-461a-a3d9-67d6ca2a91ee" containerName="payload-pre-processing"
Aug 05 18:05:48.508960 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.508753    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8347a728-4b63-461a-a3d9-67d6ca2a91ee" containerName="payload-pre-processing"
Aug 05 18:05:48.508960 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.508760    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4747837-a5b0-4a32-a3ba-d3fdd9a3c12c" containerName="payload-processing"
Aug 05 18:05:48.508960 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.508770    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d9155c5-dbfd-468a-b1df-9b76995b1cfd" containerName="maas-api"
Aug 05 18:05:48.512911 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.512890    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"
Aug 05 18:05:48.516208 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.516171    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jdhhr\""
Aug 05 18:05:48.516439 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.516424    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-65feef49-serving-cert\""
Aug 05 18:05:48.523737 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.523714    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"]
Aug 05 18:05:48.552818 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.552787    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:48.581811 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.581782    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5-openshift-service-ca\") pod \"maas-api-e2e-derive-65feef49-786bfc449b-6wsmv\" (UID: \"4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"
Aug 05 18:05:48.581912 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.581815    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5-maas-api-tls\") pod \"maas-api-e2e-derive-65feef49-786bfc449b-6wsmv\" (UID: \"4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"
Aug 05 18:05:48.581912 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.581897    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnx2f\" (UniqueName: \"kubernetes.io/projected/4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5-kube-api-access-gnx2f\") pod \"maas-api-e2e-derive-65feef49-786bfc449b-6wsmv\" (UID: \"4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"
Aug 05 18:05:48.682307 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.682280    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnx2f\" (UniqueName: \"kubernetes.io/projected/4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5-kube-api-access-gnx2f\") pod \"maas-api-e2e-derive-65feef49-786bfc449b-6wsmv\" (UID: \"4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"
Aug 05 18:05:48.682459 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.682339    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5-openshift-service-ca\") pod \"maas-api-e2e-derive-65feef49-786bfc449b-6wsmv\" (UID: \"4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"
Aug 05 18:05:48.682459 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.682361    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5-maas-api-tls\") pod \"maas-api-e2e-derive-65feef49-786bfc449b-6wsmv\" (UID: \"4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"
Aug 05 18:05:48.682977 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.682953    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5-openshift-service-ca\") pod \"maas-api-e2e-derive-65feef49-786bfc449b-6wsmv\" (UID: \"4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"
Aug 05 18:05:48.685071 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.685048    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5-maas-api-tls\") pod \"maas-api-e2e-derive-65feef49-786bfc449b-6wsmv\" (UID: \"4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"
Aug 05 18:05:48.690245 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.690221    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:48.692474 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.692452    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnx2f\" (UniqueName: \"kubernetes.io/projected/4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5-kube-api-access-gnx2f\") pod \"maas-api-e2e-derive-65feef49-786bfc449b-6wsmv\" (UID: \"4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"
Aug 05 18:05:48.824340 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.824311    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"
Aug 05 18:05:48.967144 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:48.967121    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"]
Aug 05 18:05:48.969698 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:05:48.969666    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b0dcfc3_b1e1_4b5b_b475_cbe109c4f7d5.slice/crio-27b2f15b2e099d8c821f600cc17362c6fca2dcb87b54d09e9ca1e1f236bf1bf6 WatchSource:0}: Error finding container 27b2f15b2e099d8c821f600cc17362c6fca2dcb87b54d09e9ca1e1f236bf1bf6: Status 404 returned error can't find the container with id 27b2f15b2e099d8c821f600cc17362c6fca2dcb87b54d09e9ca1e1f236bf1bf6
Aug 05 18:05:49.038140 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.038110    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:49.044976 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.044941    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv" event={"ID":"4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5","Type":"ContainerStarted","Data":"27b2f15b2e099d8c821f600cc17362c6fca2dcb87b54d09e9ca1e1f236bf1bf6"}
Aug 05 18:05:49.132733 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.132655    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:49.153607 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.153582    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b"]
Aug 05 18:05:49.158431 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.158412    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b"
Aug 05 18:05:49.161324 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.161303    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-65feef49-dockercfg-86fsr\""
Aug 05 18:05:49.161566 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.161553    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-65feef49\""
Aug 05 18:05:49.170278 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.170259    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b"]
Aug 05 18:05:49.177758 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.177734    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt"]
Aug 05 18:05:49.182771 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.182753    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt"
Aug 05 18:05:49.187292 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.187268    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fpls\" (UniqueName: \"kubernetes.io/projected/b284f81a-e9a8-4c3c-b8df-73385b773b5d-kube-api-access-8fpls\") pod \"payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b\" (UID: \"b284f81a-e9a8-4c3c-b8df-73385b773b5d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b"
Aug 05 18:05:49.187407 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.187372    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b284f81a-e9a8-4c3c-b8df-73385b773b5d-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b\" (UID: \"b284f81a-e9a8-4c3c-b8df-73385b773b5d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b"
Aug 05 18:05:49.195632 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.195611    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt"]
Aug 05 18:05:49.288275 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.288239    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffa57f35-1460-4f47-b46c-964ee74dc408-plugins-config-volume\") pod \"payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt\" (UID: \"ffa57f35-1460-4f47-b46c-964ee74dc408\") " pod="openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt"
Aug 05 18:05:49.288424 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.288403    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fpls\" (UniqueName: \"kubernetes.io/projected/b284f81a-e9a8-4c3c-b8df-73385b773b5d-kube-api-access-8fpls\") pod \"payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b\" (UID: \"b284f81a-e9a8-4c3c-b8df-73385b773b5d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b"
Aug 05 18:05:49.288489 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.288473    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b284f81a-e9a8-4c3c-b8df-73385b773b5d-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b\" (UID: \"b284f81a-e9a8-4c3c-b8df-73385b773b5d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b"
Aug 05 18:05:49.288545 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.288515    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc27b\" (UniqueName: \"kubernetes.io/projected/ffa57f35-1460-4f47-b46c-964ee74dc408-kube-api-access-gc27b\") pod \"payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt\" (UID: \"ffa57f35-1460-4f47-b46c-964ee74dc408\") " pod="openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt"
Aug 05 18:05:49.289239 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.289213    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b284f81a-e9a8-4c3c-b8df-73385b773b5d-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b\" (UID: \"b284f81a-e9a8-4c3c-b8df-73385b773b5d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b"
Aug 05 18:05:49.298912 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.298891    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fpls\" (UniqueName: \"kubernetes.io/projected/b284f81a-e9a8-4c3c-b8df-73385b773b5d-kube-api-access-8fpls\") pod \"payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b\" (UID: \"b284f81a-e9a8-4c3c-b8df-73385b773b5d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b"
Aug 05 18:05:49.389575 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.389503    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc27b\" (UniqueName: \"kubernetes.io/projected/ffa57f35-1460-4f47-b46c-964ee74dc408-kube-api-access-gc27b\") pod \"payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt\" (UID: \"ffa57f35-1460-4f47-b46c-964ee74dc408\") " pod="openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt"
Aug 05 18:05:49.389575 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.389549    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffa57f35-1460-4f47-b46c-964ee74dc408-plugins-config-volume\") pod \"payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt\" (UID: \"ffa57f35-1460-4f47-b46c-964ee74dc408\") " pod="openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt"
Aug 05 18:05:49.390204 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.390162    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffa57f35-1460-4f47-b46c-964ee74dc408-plugins-config-volume\") pod \"payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt\" (UID: \"ffa57f35-1460-4f47-b46c-964ee74dc408\") " pod="openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt"
Aug 05 18:05:49.404538 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.404513    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc27b\" (UniqueName: \"kubernetes.io/projected/ffa57f35-1460-4f47-b46c-964ee74dc408-kube-api-access-gc27b\") pod \"payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt\" (UID: \"ffa57f35-1460-4f47-b46c-964ee74dc408\") " pod="openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt"
Aug 05 18:05:49.470326 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.470298    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b"
Aug 05 18:05:49.493030 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.493004    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt"
Aug 05 18:05:49.650760 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.650731    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b"]
Aug 05 18:05:49.653579 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:05:49.653546    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb284f81a_e9a8_4c3c_b8df_73385b773b5d.slice/crio-2339e4262f6d0c48a42df7093ada264a3c9925ba1706f37b966cea0aae990685 WatchSource:0}: Error finding container 2339e4262f6d0c48a42df7093ada264a3c9925ba1706f37b966cea0aae990685: Status 404 returned error can't find the container with id 2339e4262f6d0c48a42df7093ada264a3c9925ba1706f37b966cea0aae990685
Aug 05 18:05:49.897827 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:49.897803    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt"]
Aug 05 18:05:49.899641 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:05:49.899613    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffa57f35_1460_4f47_b46c_964ee74dc408.slice/crio-a8353c9417a7ee824be5596045b5cde052ddc9ece39de752856ca9c349765f6e WatchSource:0}: Error finding container a8353c9417a7ee824be5596045b5cde052ddc9ece39de752856ca9c349765f6e: Status 404 returned error can't find the container with id a8353c9417a7ee824be5596045b5cde052ddc9ece39de752856ca9c349765f6e
Aug 05 18:05:50.054021 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:50.053989    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv" event={"ID":"4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5","Type":"ContainerStarted","Data":"602c7f61ace8c17955f5dc7de12567e756c122968e9d5f71be9163277874984a"}
Aug 05 18:05:50.054222 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:50.054135    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"
Aug 05 18:05:50.055750 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:50.055719    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt" event={"ID":"ffa57f35-1460-4f47-b46c-964ee74dc408","Type":"ContainerStarted","Data":"b106b2192a3f545250174bca5503c221eb156e31241ad781dfe2d536604def82"}
Aug 05 18:05:50.055878 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:50.055772    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt"
Aug 05 18:05:50.055878 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:50.055798    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt" event={"ID":"ffa57f35-1460-4f47-b46c-964ee74dc408","Type":"ContainerStarted","Data":"a8353c9417a7ee824be5596045b5cde052ddc9ece39de752856ca9c349765f6e"}
Aug 05 18:05:50.057654 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:50.057632    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b" event={"ID":"b284f81a-e9a8-4c3c-b8df-73385b773b5d","Type":"ContainerStarted","Data":"1869aaf7933511473b674990166dc2831b1afdf2186d2536623701234fa60d62"}
Aug 05 18:05:50.057770 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:50.057668    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b" event={"ID":"b284f81a-e9a8-4c3c-b8df-73385b773b5d","Type":"ContainerStarted","Data":"2339e4262f6d0c48a42df7093ada264a3c9925ba1706f37b966cea0aae990685"}
Aug 05 18:05:50.057770 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:50.057696    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b"
Aug 05 18:05:50.082528 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:50.082478    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv" podStartSLOduration=1.561059761 podStartE2EDuration="2.082464483s" totalImagesPullingTime="521.404722ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:05:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:05:50.079724086 +0000 UTC m=+1957.432398702" watchObservedRunningTime="2026-08-05 18:05:50.082464483 +0000 UTC m=+1957.435139076"
Aug 05 18:05:50.104026 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:50.103988    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt" podStartSLOduration=1.103974721 podStartE2EDuration="1.103974721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:05:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:05:50.102282792 +0000 UTC m=+1957.454957394" watchObservedRunningTime="2026-08-05 18:05:50.103974721 +0000 UTC m=+1957.456649314"
Aug 05 18:05:50.127118 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:50.127075    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b" podStartSLOduration=1.12706587 podStartE2EDuration="1.12706587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:05:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:05:50.125522037 +0000 UTC m=+1957.478196631" watchObservedRunningTime="2026-08-05 18:05:50.12706587 +0000 UTC m=+1957.479740463"
Aug 05 18:05:56.067878 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:05:56.067844    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"
Aug 05 18:06:01.064883 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:01.064850    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b"
Aug 05 18:06:01.065359 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:01.065169    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt"
Aug 05 18:06:19.014073 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.014036    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"]
Aug 05 18:06:19.014651 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.014442    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv" podUID="4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5" containerName="maas-api" containerID="cri-o://602c7f61ace8c17955f5dc7de12567e756c122968e9d5f71be9163277874984a" gracePeriod=30
Aug 05 18:06:19.064417 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.064385    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:19.103942 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.103906    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt"]
Aug 05 18:06:19.104211 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.104147    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt" podUID="ffa57f35-1460-4f47-b46c-964ee74dc408" containerName="payload-processing" containerID="cri-o://b106b2192a3f545250174bca5503c221eb156e31241ad781dfe2d536604def82" gracePeriod=30
Aug 05 18:06:19.114158 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.114133    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b"]
Aug 05 18:06:19.114408 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.114385    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b" podUID="b284f81a-e9a8-4c3c-b8df-73385b773b5d" containerName="payload-pre-processing" containerID="cri-o://1869aaf7933511473b674990166dc2831b1afdf2186d2536623701234fa60d62" gracePeriod=30
Aug 05 18:06:19.162547 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.162505    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:19.205268 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.205237    2569 generic.go:356] "Generic (PLEG): container finished" podID="4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5" containerID="602c7f61ace8c17955f5dc7de12567e756c122968e9d5f71be9163277874984a" exitCode=0
Aug 05 18:06:19.205405 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.205323    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv" event={"ID":"4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5","Type":"ContainerDied","Data":"602c7f61ace8c17955f5dc7de12567e756c122968e9d5f71be9163277874984a"}
Aug 05 18:06:19.258708 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.258670    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:19.351675 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.351600    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:19.384423 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.383034    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"
Aug 05 18:06:19.405973 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.405906    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b"
Aug 05 18:06:19.411432 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.411404    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt"
Aug 05 18:06:19.437008 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.436981    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:19.479888 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.479859    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5-maas-api-tls\") pod \"4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5\" (UID: \"4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5\") "
Aug 05 18:06:19.480053 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.479902    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5-openshift-service-ca\") pod \"4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5\" (UID: \"4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5\") "
Aug 05 18:06:19.480053 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.479980    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnx2f\" (UniqueName: \"kubernetes.io/projected/4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5-kube-api-access-gnx2f\") pod \"4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5\" (UID: \"4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5\") "
Aug 05 18:06:19.480444 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.480413    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5" (UID: "4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:06:19.482242 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.482215    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5" (UID: "4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:06:19.482324 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.482260    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5-kube-api-access-gnx2f" (OuterVolumeSpecName: "kube-api-access-gnx2f") pod "4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5" (UID: "4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5"). InnerVolumeSpecName "kube-api-access-gnx2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:06:19.580840 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.580816    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b284f81a-e9a8-4c3c-b8df-73385b773b5d-plugins-config-volume\") pod \"b284f81a-e9a8-4c3c-b8df-73385b773b5d\" (UID: \"b284f81a-e9a8-4c3c-b8df-73385b773b5d\") "
Aug 05 18:06:19.581020 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.580849    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gc27b\" (UniqueName: \"kubernetes.io/projected/ffa57f35-1460-4f47-b46c-964ee74dc408-kube-api-access-gc27b\") pod \"ffa57f35-1460-4f47-b46c-964ee74dc408\" (UID: \"ffa57f35-1460-4f47-b46c-964ee74dc408\") "
Aug 05 18:06:19.581020 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.580887    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffa57f35-1460-4f47-b46c-964ee74dc408-plugins-config-volume\") pod \"ffa57f35-1460-4f47-b46c-964ee74dc408\" (UID: \"ffa57f35-1460-4f47-b46c-964ee74dc408\") "
Aug 05 18:06:19.581020 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.580993    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fpls\" (UniqueName: \"kubernetes.io/projected/b284f81a-e9a8-4c3c-b8df-73385b773b5d-kube-api-access-8fpls\") pod \"b284f81a-e9a8-4c3c-b8df-73385b773b5d\" (UID: \"b284f81a-e9a8-4c3c-b8df-73385b773b5d\") "
Aug 05 18:06:19.581263 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.581234    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b284f81a-e9a8-4c3c-b8df-73385b773b5d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b284f81a-e9a8-4c3c-b8df-73385b773b5d" (UID: "b284f81a-e9a8-4c3c-b8df-73385b773b5d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:06:19.581400 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.581265    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffa57f35-1460-4f47-b46c-964ee74dc408-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ffa57f35-1460-4f47-b46c-964ee74dc408" (UID: "ffa57f35-1460-4f47-b46c-964ee74dc408"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:06:19.581400 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.581379    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnx2f\" (UniqueName: \"kubernetes.io/projected/4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5-kube-api-access-gnx2f\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:06:19.581400 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.581394    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5-maas-api-tls\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:06:19.581538 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.581402    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5-openshift-service-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:06:19.581538 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.581411    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b284f81a-e9a8-4c3c-b8df-73385b773b5d-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:06:19.581538 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.581419    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffa57f35-1460-4f47-b46c-964ee74dc408-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:06:19.582906 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.582888    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffa57f35-1460-4f47-b46c-964ee74dc408-kube-api-access-gc27b" (OuterVolumeSpecName: "kube-api-access-gc27b") pod "ffa57f35-1460-4f47-b46c-964ee74dc408" (UID: "ffa57f35-1460-4f47-b46c-964ee74dc408"). InnerVolumeSpecName "kube-api-access-gc27b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:06:19.583044 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.583029    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b284f81a-e9a8-4c3c-b8df-73385b773b5d-kube-api-access-8fpls" (OuterVolumeSpecName: "kube-api-access-8fpls") pod "b284f81a-e9a8-4c3c-b8df-73385b773b5d" (UID: "b284f81a-e9a8-4c3c-b8df-73385b773b5d"). InnerVolumeSpecName "kube-api-access-8fpls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:06:19.682299 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.682258    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gc27b\" (UniqueName: \"kubernetes.io/projected/ffa57f35-1460-4f47-b46c-964ee74dc408-kube-api-access-gc27b\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:06:19.682299 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.682295    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fpls\" (UniqueName: \"kubernetes.io/projected/b284f81a-e9a8-4c3c-b8df-73385b773b5d-kube-api-access-8fpls\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:06:19.687093 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:19.687060    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:20.123216 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.123160    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:20.211632 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.211598    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"
Aug 05 18:06:20.211810 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.211601    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv" event={"ID":"4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5","Type":"ContainerDied","Data":"27b2f15b2e099d8c821f600cc17362c6fca2dcb87b54d09e9ca1e1f236bf1bf6"}
Aug 05 18:06:20.211810 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.211729    2569 scope.go:117] "RemoveContainer" containerID="602c7f61ace8c17955f5dc7de12567e756c122968e9d5f71be9163277874984a"
Aug 05 18:06:20.212984 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.212889    2569 generic.go:356] "Generic (PLEG): container finished" podID="ffa57f35-1460-4f47-b46c-964ee74dc408" containerID="b106b2192a3f545250174bca5503c221eb156e31241ad781dfe2d536604def82" exitCode=0
Aug 05 18:06:20.212984 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.212934    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt"
Aug 05 18:06:20.212984 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.212966    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt" event={"ID":"ffa57f35-1460-4f47-b46c-964ee74dc408","Type":"ContainerDied","Data":"b106b2192a3f545250174bca5503c221eb156e31241ad781dfe2d536604def82"}
Aug 05 18:06:20.213154 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.213019    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt" event={"ID":"ffa57f35-1460-4f47-b46c-964ee74dc408","Type":"ContainerDied","Data":"a8353c9417a7ee824be5596045b5cde052ddc9ece39de752856ca9c349765f6e"}
Aug 05 18:06:20.214249 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.214223    2569 generic.go:356] "Generic (PLEG): container finished" podID="b284f81a-e9a8-4c3c-b8df-73385b773b5d" containerID="1869aaf7933511473b674990166dc2831b1afdf2186d2536623701234fa60d62" exitCode=0
Aug 05 18:06:20.214361 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.214260    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b" event={"ID":"b284f81a-e9a8-4c3c-b8df-73385b773b5d","Type":"ContainerDied","Data":"1869aaf7933511473b674990166dc2831b1afdf2186d2536623701234fa60d62"}
Aug 05 18:06:20.214361 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.214296    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b" event={"ID":"b284f81a-e9a8-4c3c-b8df-73385b773b5d","Type":"ContainerDied","Data":"2339e4262f6d0c48a42df7093ada264a3c9925ba1706f37b966cea0aae990685"}
Aug 05 18:06:20.214361 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.214263    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b"
Aug 05 18:06:20.222910 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.222891    2569 scope.go:117] "RemoveContainer" containerID="b106b2192a3f545250174bca5503c221eb156e31241ad781dfe2d536604def82"
Aug 05 18:06:20.231139 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.231123    2569 scope.go:117] "RemoveContainer" containerID="b106b2192a3f545250174bca5503c221eb156e31241ad781dfe2d536604def82"
Aug 05 18:06:20.231464 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:06:20.231446    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b106b2192a3f545250174bca5503c221eb156e31241ad781dfe2d536604def82\": container with ID starting with b106b2192a3f545250174bca5503c221eb156e31241ad781dfe2d536604def82 not found: ID does not exist" containerID="b106b2192a3f545250174bca5503c221eb156e31241ad781dfe2d536604def82"
Aug 05 18:06:20.231525 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.231471    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b106b2192a3f545250174bca5503c221eb156e31241ad781dfe2d536604def82"} err="failed to get container status \"b106b2192a3f545250174bca5503c221eb156e31241ad781dfe2d536604def82\": rpc error: code = NotFound desc = could not find container \"b106b2192a3f545250174bca5503c221eb156e31241ad781dfe2d536604def82\": container with ID starting with b106b2192a3f545250174bca5503c221eb156e31241ad781dfe2d536604def82 not found: ID does not exist"
Aug 05 18:06:20.231525 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.231488    2569 scope.go:117] "RemoveContainer" containerID="1869aaf7933511473b674990166dc2831b1afdf2186d2536623701234fa60d62"
Aug 05 18:06:20.240438 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.240418    2569 scope.go:117] "RemoveContainer" containerID="1869aaf7933511473b674990166dc2831b1afdf2186d2536623701234fa60d62"
Aug 05 18:06:20.240737 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:06:20.240710    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1869aaf7933511473b674990166dc2831b1afdf2186d2536623701234fa60d62\": container with ID starting with 1869aaf7933511473b674990166dc2831b1afdf2186d2536623701234fa60d62 not found: ID does not exist" containerID="1869aaf7933511473b674990166dc2831b1afdf2186d2536623701234fa60d62"
Aug 05 18:06:20.240856 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.240743    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1869aaf7933511473b674990166dc2831b1afdf2186d2536623701234fa60d62"} err="failed to get container status \"1869aaf7933511473b674990166dc2831b1afdf2186d2536623701234fa60d62\": rpc error: code = NotFound desc = could not find container \"1869aaf7933511473b674990166dc2831b1afdf2186d2536623701234fa60d62\": container with ID starting with 1869aaf7933511473b674990166dc2831b1afdf2186d2536623701234fa60d62 not found: ID does not exist"
Aug 05 18:06:20.241525 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.241503    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b"]
Aug 05 18:06:20.242780 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.242752    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:20.246129 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.246111    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-65feef49-f976d9c64-djq7b"]
Aug 05 18:06:20.257983 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.257960    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt"]
Aug 05 18:06:20.261079 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.261056    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-65feef49-78c594d99d-q7mjt"]
Aug 05 18:06:20.272426 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.272406    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"]
Aug 05 18:06:20.275816 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.275797    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-65feef49-786bfc449b-6wsmv"]
Aug 05 18:06:20.340410 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:20.340383    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:21.152944 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:21.152910    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5" path="/var/lib/kubelet/pods/4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5/volumes"
Aug 05 18:06:21.153474 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:21.153452    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b284f81a-e9a8-4c3c-b8df-73385b773b5d" path="/var/lib/kubelet/pods/b284f81a-e9a8-4c3c-b8df-73385b773b5d/volumes"
Aug 05 18:06:21.153909 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:21.153893    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffa57f35-1460-4f47-b46c-964ee74dc408" path="/var/lib/kubelet/pods/ffa57f35-1460-4f47-b46c-964ee74dc408/volumes"
Aug 05 18:06:32.000121 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:32.000082    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:38.861702 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:38.861661    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:38.967936 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:38.967895    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:39.094849 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:39.094820    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:39.190039 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:39.189957    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:39.612846 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:06:39.612816    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:07:05.537346 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:07:05.537302    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:07:12.006311 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:07:12.006273    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:07:12.111496 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:07:12.111464    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:07:12.207268 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:07:12.207239    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:07:28.082377 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:07:28.082342    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:07:30.846314 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:07:30.846276    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:07:34.580418 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:07:34.580347    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:07:34.729968 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:07:34.729941    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:07:35.027128 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:07:35.026999    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:07:45.363437 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:07:45.363402    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:07:56.334891 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:07:56.334856    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:03.185625 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:03.185592    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:03.291417 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:03.291385    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:03.390005 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:03.389967    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:03.491282 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:03.491193    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:04.154565 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:04.154531    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:04.265502 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:04.265472    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:05.747542 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:05.747503    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:13.064538 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:13.064500    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:13.166626 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:13.166595    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:13.752706 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:13.752670    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:21.899226 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:21.899165    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:22.012576 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:22.012548    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:22.192884 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:22.192786    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:22.483360 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:22.483288    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:22.714702 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:22.714668    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:25.494237 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:25.494192    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:25.637500 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:25.637467    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:26.593046 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.593008    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"]
Aug 05 18:08:26.593609 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.593589    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5" containerName="maas-api"
Aug 05 18:08:26.593700 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.593611    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5" containerName="maas-api"
Aug 05 18:08:26.593700 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.593623    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffa57f35-1460-4f47-b46c-964ee74dc408" containerName="payload-processing"
Aug 05 18:08:26.593700 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.593630    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffa57f35-1460-4f47-b46c-964ee74dc408" containerName="payload-processing"
Aug 05 18:08:26.593700 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.593661    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b284f81a-e9a8-4c3c-b8df-73385b773b5d" containerName="payload-pre-processing"
Aug 05 18:08:26.593700 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.593672    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b284f81a-e9a8-4c3c-b8df-73385b773b5d" containerName="payload-pre-processing"
Aug 05 18:08:26.593950 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.593775    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffa57f35-1460-4f47-b46c-964ee74dc408" containerName="payload-processing"
Aug 05 18:08:26.593950 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.593788    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b284f81a-e9a8-4c3c-b8df-73385b773b5d" containerName="payload-pre-processing"
Aug 05 18:08:26.593950 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.593797    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b0dcfc3-b1e1-4b5b-b475-cbe109c4f7d5" containerName="maas-api"
Aug 05 18:08:26.597044 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.597024    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"
Aug 05 18:08:26.602173 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.602129    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2e99d153-serving-cert\""
Aug 05 18:08:26.602322 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.602290    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jdhhr\""
Aug 05 18:08:26.631929 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.631908    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"]
Aug 05 18:08:26.640759 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.640737    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:26.656774 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.656752    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bswfc\" (UniqueName: \"kubernetes.io/projected/43768536-9b9f-4966-97ca-e8de5bbdb564-kube-api-access-bswfc\") pod \"maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b\" (UID: \"43768536-9b9f-4966-97ca-e8de5bbdb564\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"
Aug 05 18:08:26.656878 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.656802    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43768536-9b9f-4966-97ca-e8de5bbdb564-openshift-service-ca\") pod \"maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b\" (UID: \"43768536-9b9f-4966-97ca-e8de5bbdb564\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"
Aug 05 18:08:26.656979 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.656882    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43768536-9b9f-4966-97ca-e8de5bbdb564-maas-api-tls\") pod \"maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b\" (UID: \"43768536-9b9f-4966-97ca-e8de5bbdb564\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"
Aug 05 18:08:26.758138 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.758107    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43768536-9b9f-4966-97ca-e8de5bbdb564-maas-api-tls\") pod \"maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b\" (UID: \"43768536-9b9f-4966-97ca-e8de5bbdb564\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"
Aug 05 18:08:26.758295 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.758224    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bswfc\" (UniqueName: \"kubernetes.io/projected/43768536-9b9f-4966-97ca-e8de5bbdb564-kube-api-access-bswfc\") pod \"maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b\" (UID: \"43768536-9b9f-4966-97ca-e8de5bbdb564\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"
Aug 05 18:08:26.758295 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.758257    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43768536-9b9f-4966-97ca-e8de5bbdb564-openshift-service-ca\") pod \"maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b\" (UID: \"43768536-9b9f-4966-97ca-e8de5bbdb564\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"
Aug 05 18:08:26.758295 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:08:26.758259    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-serving-cert: secret "maas-api-e2e-mt-2e99d153-serving-cert" not found
Aug 05 18:08:26.758463 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:08:26.758326    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43768536-9b9f-4966-97ca-e8de5bbdb564-maas-api-tls podName:43768536-9b9f-4966-97ca-e8de5bbdb564 nodeName:}" failed. No retries permitted until 2026-08-05 18:08:27.258309026 +0000 UTC m=+2114.610983601 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/43768536-9b9f-4966-97ca-e8de5bbdb564-maas-api-tls") pod "maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b" (UID: "43768536-9b9f-4966-97ca-e8de5bbdb564") : secret "maas-api-e2e-mt-2e99d153-serving-cert" not found
Aug 05 18:08:26.758827 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.758811    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43768536-9b9f-4966-97ca-e8de5bbdb564-openshift-service-ca\") pod \"maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b\" (UID: \"43768536-9b9f-4966-97ca-e8de5bbdb564\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"
Aug 05 18:08:26.769598 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:26.769574    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bswfc\" (UniqueName: \"kubernetes.io/projected/43768536-9b9f-4966-97ca-e8de5bbdb564-kube-api-access-bswfc\") pod \"maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b\" (UID: \"43768536-9b9f-4966-97ca-e8de5bbdb564\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"
Aug 05 18:08:27.227392 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.227358    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:27.262803 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.262776    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43768536-9b9f-4966-97ca-e8de5bbdb564-maas-api-tls\") pod \"maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b\" (UID: \"43768536-9b9f-4966-97ca-e8de5bbdb564\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"
Aug 05 18:08:27.265373 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.265340    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43768536-9b9f-4966-97ca-e8de5bbdb564-maas-api-tls\") pod \"maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b\" (UID: \"43768536-9b9f-4966-97ca-e8de5bbdb564\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"
Aug 05 18:08:27.329784 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.329749    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:27.432160 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.432130    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:27.451708 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.451677    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq"]
Aug 05 18:08:27.455361 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.455346    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq"
Aug 05 18:08:27.458297 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.458275    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-2e99d153\""
Aug 05 18:08:27.458411 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.458350    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-2e99d153-dockercfg-rlzg5\""
Aug 05 18:08:27.459740 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.459714    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h"]
Aug 05 18:08:27.465560 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.465522    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h"
Aug 05 18:08:27.474220 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.474199    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq"]
Aug 05 18:08:27.490907 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.490873    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h"]
Aug 05 18:08:27.507037 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.506916    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"
Aug 05 18:08:27.568261 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.568215    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80db473b-a120-4c89-8631-809cecd830b7-plugins-config-volume\") pod \"payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h\" (UID: \"80db473b-a120-4c89-8631-809cecd830b7\") " pod="openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h"
Aug 05 18:08:27.568427 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.568284    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dfb5\" (UniqueName: \"kubernetes.io/projected/80db473b-a120-4c89-8631-809cecd830b7-kube-api-access-6dfb5\") pod \"payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h\" (UID: \"80db473b-a120-4c89-8631-809cecd830b7\") " pod="openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h"
Aug 05 18:08:27.568427 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.568380    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npf9x\" (UniqueName: \"kubernetes.io/projected/5c582154-cd7d-4517-83b2-bec670979d41-kube-api-access-npf9x\") pod \"payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq\" (UID: \"5c582154-cd7d-4517-83b2-bec670979d41\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq"
Aug 05 18:08:27.568427 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.568419    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c582154-cd7d-4517-83b2-bec670979d41-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq\" (UID: \"5c582154-cd7d-4517-83b2-bec670979d41\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq"
Aug 05 18:08:27.656846 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.656818    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"]
Aug 05 18:08:27.658862 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:08:27.658835    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43768536_9b9f_4966_97ca_e8de5bbdb564.slice/crio-1002db9a28cb378ff2260436fb0dab44cfcdd92b058ffbd4187a984fe8bc818e WatchSource:0}: Error finding container 1002db9a28cb378ff2260436fb0dab44cfcdd92b058ffbd4187a984fe8bc818e: Status 404 returned error can't find the container with id 1002db9a28cb378ff2260436fb0dab44cfcdd92b058ffbd4187a984fe8bc818e
Aug 05 18:08:27.669392 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.669368    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80db473b-a120-4c89-8631-809cecd830b7-plugins-config-volume\") pod \"payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h\" (UID: \"80db473b-a120-4c89-8631-809cecd830b7\") " pod="openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h"
Aug 05 18:08:27.669503 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.669429    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dfb5\" (UniqueName: \"kubernetes.io/projected/80db473b-a120-4c89-8631-809cecd830b7-kube-api-access-6dfb5\") pod \"payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h\" (UID: \"80db473b-a120-4c89-8631-809cecd830b7\") " pod="openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h"
Aug 05 18:08:27.669573 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.669511    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npf9x\" (UniqueName: \"kubernetes.io/projected/5c582154-cd7d-4517-83b2-bec670979d41-kube-api-access-npf9x\") pod \"payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq\" (UID: \"5c582154-cd7d-4517-83b2-bec670979d41\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq"
Aug 05 18:08:27.669573 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.669555    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c582154-cd7d-4517-83b2-bec670979d41-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq\" (UID: \"5c582154-cd7d-4517-83b2-bec670979d41\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq"
Aug 05 18:08:27.670032 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.670008    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80db473b-a120-4c89-8631-809cecd830b7-plugins-config-volume\") pod \"payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h\" (UID: \"80db473b-a120-4c89-8631-809cecd830b7\") " pod="openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h"
Aug 05 18:08:27.670223 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.670205    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c582154-cd7d-4517-83b2-bec670979d41-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq\" (UID: \"5c582154-cd7d-4517-83b2-bec670979d41\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq"
Aug 05 18:08:27.678648 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.678630    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dfb5\" (UniqueName: \"kubernetes.io/projected/80db473b-a120-4c89-8631-809cecd830b7-kube-api-access-6dfb5\") pod \"payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h\" (UID: \"80db473b-a120-4c89-8631-809cecd830b7\") " pod="openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h"
Aug 05 18:08:27.678728 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.678692    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npf9x\" (UniqueName: \"kubernetes.io/projected/5c582154-cd7d-4517-83b2-bec670979d41-kube-api-access-npf9x\") pod \"payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq\" (UID: \"5c582154-cd7d-4517-83b2-bec670979d41\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq"
Aug 05 18:08:27.779325 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.779265    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq"
Aug 05 18:08:27.788230 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.787987    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h"
Aug 05 18:08:27.886570 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.886541    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b" event={"ID":"43768536-9b9f-4966-97ca-e8de5bbdb564","Type":"ContainerStarted","Data":"1002db9a28cb378ff2260436fb0dab44cfcdd92b058ffbd4187a984fe8bc818e"}
Aug 05 18:08:27.947384 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.947350    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq"]
Aug 05 18:08:27.951149 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:08:27.951118    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c582154_cd7d_4517_83b2_bec670979d41.slice/crio-e6b594f195ef7adc6549003f55d9a99390b442a8320839f9b6befbc4566a6ba4 WatchSource:0}: Error finding container e6b594f195ef7adc6549003f55d9a99390b442a8320839f9b6befbc4566a6ba4: Status 404 returned error can't find the container with id e6b594f195ef7adc6549003f55d9a99390b442a8320839f9b6befbc4566a6ba4
Aug 05 18:08:27.963192 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:27.963149    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h"]
Aug 05 18:08:27.965360 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:08:27.965338    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80db473b_a120_4c89_8631_809cecd830b7.slice/crio-9cbe80a56e32349ef463c1a22b30a79580a2a67b70e14459987309591d258edc WatchSource:0}: Error finding container 9cbe80a56e32349ef463c1a22b30a79580a2a67b70e14459987309591d258edc: Status 404 returned error can't find the container with id 9cbe80a56e32349ef463c1a22b30a79580a2a67b70e14459987309591d258edc
Aug 05 18:08:28.893701 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:28.893670    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h" event={"ID":"80db473b-a120-4c89-8631-809cecd830b7","Type":"ContainerStarted","Data":"cfafc12895cedb9dbd72388e23084b230ce82bdebd883af233256900e138e368"}
Aug 05 18:08:28.894146 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:28.893721    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h" event={"ID":"80db473b-a120-4c89-8631-809cecd830b7","Type":"ContainerStarted","Data":"9cbe80a56e32349ef463c1a22b30a79580a2a67b70e14459987309591d258edc"}
Aug 05 18:08:28.894146 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:28.893783    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h"
Aug 05 18:08:28.895159 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:28.895136    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq" event={"ID":"5c582154-cd7d-4517-83b2-bec670979d41","Type":"ContainerStarted","Data":"1251224b424f7032bca163deb79363523eaf0ac1759ec629c3e7bb28bf101e54"}
Aug 05 18:08:28.895300 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:28.895202    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq"
Aug 05 18:08:28.895300 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:28.895223    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq" event={"ID":"5c582154-cd7d-4517-83b2-bec670979d41","Type":"ContainerStarted","Data":"e6b594f195ef7adc6549003f55d9a99390b442a8320839f9b6befbc4566a6ba4"}
Aug 05 18:08:28.896527 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:28.896504    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b" event={"ID":"43768536-9b9f-4966-97ca-e8de5bbdb564","Type":"ContainerStarted","Data":"673004438a61757f2d030e8870a25ab26d649c3e844947ad76a2c11cdb2c41b6"}
Aug 05 18:08:28.896665 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:28.896648    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"
Aug 05 18:08:28.914154 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:28.914112    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h" podStartSLOduration=1.9141000940000001 podStartE2EDuration="1.914100094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:08:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:08:28.912408658 +0000 UTC m=+2116.265083255" watchObservedRunningTime="2026-08-05 18:08:28.914100094 +0000 UTC m=+2116.266774688"
Aug 05 18:08:28.934407 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:28.934365    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq" podStartSLOduration=1.934352778 podStartE2EDuration="1.934352778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:08:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:08:28.932335757 +0000 UTC m=+2116.285010350" watchObservedRunningTime="2026-08-05 18:08:28.934352778 +0000 UTC m=+2116.287027371"
Aug 05 18:08:28.955006 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:28.954967    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b" podStartSLOduration=2.489827063 podStartE2EDuration="2.954953703s" totalImagesPullingTime="465.12664ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:08:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:08:28.9534158 +0000 UTC m=+2116.306090420" watchObservedRunningTime="2026-08-05 18:08:28.954953703 +0000 UTC m=+2116.307628296"
Aug 05 18:08:34.906712 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:34.906683    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"
Aug 05 18:08:39.902467 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:39.902421    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq" podUID="5c582154-cd7d-4517-83b2-bec670979d41" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.59:9004: connect: connection refused"
Aug 05 18:08:39.902904 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:39.902817    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h"
Aug 05 18:08:49.902174 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:49.902142    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq"
Aug 05 18:08:50.432326 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:50.432290    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:51.055566 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:51.055526    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:51.161121 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:51.161091    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:51.518319 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:51.518283    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:55.082507 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:55.082467    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:57.725581 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:57.725543    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:57.828233 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:08:57.828200    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:08.680425 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:08.680341    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"]
Aug 05 18:09:08.680906 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:08.680542    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b" podUID="43768536-9b9f-4966-97ca-e8de5bbdb564" containerName="maas-api" containerID="cri-o://673004438a61757f2d030e8870a25ab26d649c3e844947ad76a2c11cdb2c41b6" gracePeriod=30
Aug 05 18:09:08.725941 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:08.725914    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h"]
Aug 05 18:09:08.726162 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:08.726138    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h" podUID="80db473b-a120-4c89-8631-809cecd830b7" containerName="payload-processing" containerID="cri-o://cfafc12895cedb9dbd72388e23084b230ce82bdebd883af233256900e138e368" gracePeriod=30
Aug 05 18:09:08.737087 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:08.737059    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq"]
Aug 05 18:09:08.737581 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:08.737552    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq" podUID="5c582154-cd7d-4517-83b2-bec670979d41" containerName="payload-pre-processing" containerID="cri-o://1251224b424f7032bca163deb79363523eaf0ac1759ec629c3e7bb28bf101e54" gracePeriod=30
Aug 05 18:09:08.772535 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:08.772513    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:08.873013 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:08.872970    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:08.948603 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:08.948573    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:09.023977 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.023954    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h"
Aug 05 18:09:09.027382 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.027362    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"
Aug 05 18:09:09.030781 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.030761    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq"
Aug 05 18:09:09.053051 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.053030    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c582154-cd7d-4517-83b2-bec670979d41-plugins-config-volume\") pod \"5c582154-cd7d-4517-83b2-bec670979d41\" (UID: \"5c582154-cd7d-4517-83b2-bec670979d41\") "
Aug 05 18:09:09.053171 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.053078    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bswfc\" (UniqueName: \"kubernetes.io/projected/43768536-9b9f-4966-97ca-e8de5bbdb564-kube-api-access-bswfc\") pod \"43768536-9b9f-4966-97ca-e8de5bbdb564\" (UID: \"43768536-9b9f-4966-97ca-e8de5bbdb564\") "
Aug 05 18:09:09.053171 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.053111    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80db473b-a120-4c89-8631-809cecd830b7-plugins-config-volume\") pod \"80db473b-a120-4c89-8631-809cecd830b7\" (UID: \"80db473b-a120-4c89-8631-809cecd830b7\") "
Aug 05 18:09:09.053171 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.053157    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npf9x\" (UniqueName: \"kubernetes.io/projected/5c582154-cd7d-4517-83b2-bec670979d41-kube-api-access-npf9x\") pod \"5c582154-cd7d-4517-83b2-bec670979d41\" (UID: \"5c582154-cd7d-4517-83b2-bec670979d41\") "
Aug 05 18:09:09.053317 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.053219    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43768536-9b9f-4966-97ca-e8de5bbdb564-openshift-service-ca\") pod \"43768536-9b9f-4966-97ca-e8de5bbdb564\" (UID: \"43768536-9b9f-4966-97ca-e8de5bbdb564\") "
Aug 05 18:09:09.053317 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.053258    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dfb5\" (UniqueName: \"kubernetes.io/projected/80db473b-a120-4c89-8631-809cecd830b7-kube-api-access-6dfb5\") pod \"80db473b-a120-4c89-8631-809cecd830b7\" (UID: \"80db473b-a120-4c89-8631-809cecd830b7\") "
Aug 05 18:09:09.053389 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.053334    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43768536-9b9f-4966-97ca-e8de5bbdb564-maas-api-tls\") pod \"43768536-9b9f-4966-97ca-e8de5bbdb564\" (UID: \"43768536-9b9f-4966-97ca-e8de5bbdb564\") "
Aug 05 18:09:09.053480 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.053459    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c582154-cd7d-4517-83b2-bec670979d41-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5c582154-cd7d-4517-83b2-bec670979d41" (UID: "5c582154-cd7d-4517-83b2-bec670979d41"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:09:09.053568 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.053512    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80db473b-a120-4c89-8631-809cecd830b7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "80db473b-a120-4c89-8631-809cecd830b7" (UID: "80db473b-a120-4c89-8631-809cecd830b7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:09:09.054073 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.053767    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43768536-9b9f-4966-97ca-e8de5bbdb564-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "43768536-9b9f-4966-97ca-e8de5bbdb564" (UID: "43768536-9b9f-4966-97ca-e8de5bbdb564"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:09:09.054073 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.053803    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80db473b-a120-4c89-8631-809cecd830b7-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:09:09.054073 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.053822    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c582154-cd7d-4517-83b2-bec670979d41-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:09:09.055700 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.055674    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43768536-9b9f-4966-97ca-e8de5bbdb564-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "43768536-9b9f-4966-97ca-e8de5bbdb564" (UID: "43768536-9b9f-4966-97ca-e8de5bbdb564"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:09:09.055906 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.055877    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43768536-9b9f-4966-97ca-e8de5bbdb564-kube-api-access-bswfc" (OuterVolumeSpecName: "kube-api-access-bswfc") pod "43768536-9b9f-4966-97ca-e8de5bbdb564" (UID: "43768536-9b9f-4966-97ca-e8de5bbdb564"). InnerVolumeSpecName "kube-api-access-bswfc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:09:09.056209 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.056159    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80db473b-a120-4c89-8631-809cecd830b7-kube-api-access-6dfb5" (OuterVolumeSpecName: "kube-api-access-6dfb5") pod "80db473b-a120-4c89-8631-809cecd830b7" (UID: "80db473b-a120-4c89-8631-809cecd830b7"). InnerVolumeSpecName "kube-api-access-6dfb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:09:09.056209 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.056159    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c582154-cd7d-4517-83b2-bec670979d41-kube-api-access-npf9x" (OuterVolumeSpecName: "kube-api-access-npf9x") pod "5c582154-cd7d-4517-83b2-bec670979d41" (UID: "5c582154-cd7d-4517-83b2-bec670979d41"). InnerVolumeSpecName "kube-api-access-npf9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:09:09.100567 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.100538    2569 generic.go:356] "Generic (PLEG): container finished" podID="80db473b-a120-4c89-8631-809cecd830b7" containerID="cfafc12895cedb9dbd72388e23084b230ce82bdebd883af233256900e138e368" exitCode=0
Aug 05 18:09:09.100726 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.100590    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h"
Aug 05 18:09:09.100726 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.100621    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h" event={"ID":"80db473b-a120-4c89-8631-809cecd830b7","Type":"ContainerDied","Data":"cfafc12895cedb9dbd72388e23084b230ce82bdebd883af233256900e138e368"}
Aug 05 18:09:09.100726 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.100686    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h" event={"ID":"80db473b-a120-4c89-8631-809cecd830b7","Type":"ContainerDied","Data":"9cbe80a56e32349ef463c1a22b30a79580a2a67b70e14459987309591d258edc"}
Aug 05 18:09:09.100726 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.100722    2569 scope.go:117] "RemoveContainer" containerID="cfafc12895cedb9dbd72388e23084b230ce82bdebd883af233256900e138e368"
Aug 05 18:09:09.102032 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.101942    2569 generic.go:356] "Generic (PLEG): container finished" podID="5c582154-cd7d-4517-83b2-bec670979d41" containerID="1251224b424f7032bca163deb79363523eaf0ac1759ec629c3e7bb28bf101e54" exitCode=0
Aug 05 18:09:09.102032 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.101982    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq"
Aug 05 18:09:09.102163 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.102039    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq" event={"ID":"5c582154-cd7d-4517-83b2-bec670979d41","Type":"ContainerDied","Data":"1251224b424f7032bca163deb79363523eaf0ac1759ec629c3e7bb28bf101e54"}
Aug 05 18:09:09.102163 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.102081    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq" event={"ID":"5c582154-cd7d-4517-83b2-bec670979d41","Type":"ContainerDied","Data":"e6b594f195ef7adc6549003f55d9a99390b442a8320839f9b6befbc4566a6ba4"}
Aug 05 18:09:09.103587 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.103561    2569 generic.go:356] "Generic (PLEG): container finished" podID="43768536-9b9f-4966-97ca-e8de5bbdb564" containerID="673004438a61757f2d030e8870a25ab26d649c3e844947ad76a2c11cdb2c41b6" exitCode=0
Aug 05 18:09:09.103718 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.103610    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"
Aug 05 18:09:09.103718 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.103659    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b" event={"ID":"43768536-9b9f-4966-97ca-e8de5bbdb564","Type":"ContainerDied","Data":"673004438a61757f2d030e8870a25ab26d649c3e844947ad76a2c11cdb2c41b6"}
Aug 05 18:09:09.103846 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.103748    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b" event={"ID":"43768536-9b9f-4966-97ca-e8de5bbdb564","Type":"ContainerDied","Data":"1002db9a28cb378ff2260436fb0dab44cfcdd92b058ffbd4187a984fe8bc818e"}
Aug 05 18:09:09.111722 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.111704    2569 scope.go:117] "RemoveContainer" containerID="cfafc12895cedb9dbd72388e23084b230ce82bdebd883af233256900e138e368"
Aug 05 18:09:09.111966 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:09:09.111935    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfafc12895cedb9dbd72388e23084b230ce82bdebd883af233256900e138e368\": container with ID starting with cfafc12895cedb9dbd72388e23084b230ce82bdebd883af233256900e138e368 not found: ID does not exist" containerID="cfafc12895cedb9dbd72388e23084b230ce82bdebd883af233256900e138e368"
Aug 05 18:09:09.112054 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.111963    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfafc12895cedb9dbd72388e23084b230ce82bdebd883af233256900e138e368"} err="failed to get container status \"cfafc12895cedb9dbd72388e23084b230ce82bdebd883af233256900e138e368\": rpc error: code = NotFound desc = could not find container \"cfafc12895cedb9dbd72388e23084b230ce82bdebd883af233256900e138e368\": container with ID starting with cfafc12895cedb9dbd72388e23084b230ce82bdebd883af233256900e138e368 not found: ID does not exist"
Aug 05 18:09:09.112054 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.111978    2569 scope.go:117] "RemoveContainer" containerID="1251224b424f7032bca163deb79363523eaf0ac1759ec629c3e7bb28bf101e54"
Aug 05 18:09:09.120348 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.120332    2569 scope.go:117] "RemoveContainer" containerID="1251224b424f7032bca163deb79363523eaf0ac1759ec629c3e7bb28bf101e54"
Aug 05 18:09:09.120580 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:09:09.120561    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1251224b424f7032bca163deb79363523eaf0ac1759ec629c3e7bb28bf101e54\": container with ID starting with 1251224b424f7032bca163deb79363523eaf0ac1759ec629c3e7bb28bf101e54 not found: ID does not exist" containerID="1251224b424f7032bca163deb79363523eaf0ac1759ec629c3e7bb28bf101e54"
Aug 05 18:09:09.120644 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.120586    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1251224b424f7032bca163deb79363523eaf0ac1759ec629c3e7bb28bf101e54"} err="failed to get container status \"1251224b424f7032bca163deb79363523eaf0ac1759ec629c3e7bb28bf101e54\": rpc error: code = NotFound desc = could not find container \"1251224b424f7032bca163deb79363523eaf0ac1759ec629c3e7bb28bf101e54\": container with ID starting with 1251224b424f7032bca163deb79363523eaf0ac1759ec629c3e7bb28bf101e54 not found: ID does not exist"
Aug 05 18:09:09.120644 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.120603    2569 scope.go:117] "RemoveContainer" containerID="673004438a61757f2d030e8870a25ab26d649c3e844947ad76a2c11cdb2c41b6"
Aug 05 18:09:09.128281 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.128258    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h"]
Aug 05 18:09:09.129587 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.129562    2569 scope.go:117] "RemoveContainer" containerID="673004438a61757f2d030e8870a25ab26d649c3e844947ad76a2c11cdb2c41b6"
Aug 05 18:09:09.129830 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:09:09.129812    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"673004438a61757f2d030e8870a25ab26d649c3e844947ad76a2c11cdb2c41b6\": container with ID starting with 673004438a61757f2d030e8870a25ab26d649c3e844947ad76a2c11cdb2c41b6 not found: ID does not exist" containerID="673004438a61757f2d030e8870a25ab26d649c3e844947ad76a2c11cdb2c41b6"
Aug 05 18:09:09.129901 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.129836    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"673004438a61757f2d030e8870a25ab26d649c3e844947ad76a2c11cdb2c41b6"} err="failed to get container status \"673004438a61757f2d030e8870a25ab26d649c3e844947ad76a2c11cdb2c41b6\": rpc error: code = NotFound desc = could not find container \"673004438a61757f2d030e8870a25ab26d649c3e844947ad76a2c11cdb2c41b6\": container with ID starting with 673004438a61757f2d030e8870a25ab26d649c3e844947ad76a2c11cdb2c41b6 not found: ID does not exist"
Aug 05 18:09:09.130216 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.130197    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2e99d153-767c9fd7bb-g2k2h"]
Aug 05 18:09:09.141636 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.141610    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq"]
Aug 05 18:09:09.143634 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.143613    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2e99d153-5c9dcbdd48-6m9cq"]
Aug 05 18:09:09.151843 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.151821    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c582154-cd7d-4517-83b2-bec670979d41" path="/var/lib/kubelet/pods/5c582154-cd7d-4517-83b2-bec670979d41/volumes"
Aug 05 18:09:09.152307 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.152291    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80db473b-a120-4c89-8631-809cecd830b7" path="/var/lib/kubelet/pods/80db473b-a120-4c89-8631-809cecd830b7/volumes"
Aug 05 18:09:09.154830 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.154811    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bswfc\" (UniqueName: \"kubernetes.io/projected/43768536-9b9f-4966-97ca-e8de5bbdb564-kube-api-access-bswfc\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:09:09.154830 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.154830    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npf9x\" (UniqueName: \"kubernetes.io/projected/5c582154-cd7d-4517-83b2-bec670979d41-kube-api-access-npf9x\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:09:09.154830 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.154840    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43768536-9b9f-4966-97ca-e8de5bbdb564-openshift-service-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:09:09.154992 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.154849    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dfb5\" (UniqueName: \"kubernetes.io/projected/80db473b-a120-4c89-8631-809cecd830b7-kube-api-access-6dfb5\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:09:09.154992 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.154859    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43768536-9b9f-4966-97ca-e8de5bbdb564-maas-api-tls\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:09:09.155557 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.155541    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"]
Aug 05 18:09:09.161061 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.161043    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2e99d153-6d7bf4b666-vr95b"]
Aug 05 18:09:09.281316 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:09.281254    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:11.150346 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:11.150313    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43768536-9b9f-4966-97ca-e8de5bbdb564" path="/var/lib/kubelet/pods/43768536-9b9f-4966-97ca-e8de5bbdb564/volumes"
Aug 05 18:09:15.986754 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:15.986713    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:16.091378 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:16.091334    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:16.303437 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:16.303351    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:16.573649 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:16.573619    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:16.787437 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:16.787388    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:17.037243 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:17.037152    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:24.154952 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:24.154917    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:24.323966 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:24.323934    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:24.467830 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:24.467713    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:24.557931 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:24.557890    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:31.200448 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:31.200393    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:31.311518 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:31.311486    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:45.657549 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:45.657497    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:59.551188 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:59.551143    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x"]
Aug 05 18:09:59.551779 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:59.551364    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x" podUID="66eb3ae5-5efe-4070-a999-d399e0a875f0" containerName="manager" containerID="cri-o://b4dd36160c6884d19da84288cb730f2273d3b7ca0974d7d47296084893bc1373" gracePeriod=10
Aug 05 18:09:59.800934 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:59.800912    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x"
Aug 05 18:09:59.932893 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:59.932869    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfj7j\" (UniqueName: \"kubernetes.io/projected/66eb3ae5-5efe-4070-a999-d399e0a875f0-kube-api-access-mfj7j\") pod \"66eb3ae5-5efe-4070-a999-d399e0a875f0\" (UID: \"66eb3ae5-5efe-4070-a999-d399e0a875f0\") "
Aug 05 18:09:59.933070 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:59.932960    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/66eb3ae5-5efe-4070-a999-d399e0a875f0-extensions-socket-volume\") pod \"66eb3ae5-5efe-4070-a999-d399e0a875f0\" (UID: \"66eb3ae5-5efe-4070-a999-d399e0a875f0\") "
Aug 05 18:09:59.933371 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:59.933343    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66eb3ae5-5efe-4070-a999-d399e0a875f0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "66eb3ae5-5efe-4070-a999-d399e0a875f0" (UID: "66eb3ae5-5efe-4070-a999-d399e0a875f0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:09:59.935034 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:09:59.935007    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66eb3ae5-5efe-4070-a999-d399e0a875f0-kube-api-access-mfj7j" (OuterVolumeSpecName: "kube-api-access-mfj7j") pod "66eb3ae5-5efe-4070-a999-d399e0a875f0" (UID: "66eb3ae5-5efe-4070-a999-d399e0a875f0"). InnerVolumeSpecName "kube-api-access-mfj7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:10:00.034150 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:10:00.034124    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfj7j\" (UniqueName: \"kubernetes.io/projected/66eb3ae5-5efe-4070-a999-d399e0a875f0-kube-api-access-mfj7j\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:10:00.034150 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:10:00.034150    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/66eb3ae5-5efe-4070-a999-d399e0a875f0-extensions-socket-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:10:00.361879 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:10:00.361847    2569 generic.go:356] "Generic (PLEG): container finished" podID="66eb3ae5-5efe-4070-a999-d399e0a875f0" containerID="b4dd36160c6884d19da84288cb730f2273d3b7ca0974d7d47296084893bc1373" exitCode=0
Aug 05 18:10:00.362039 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:10:00.361897    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x"
Aug 05 18:10:00.362039 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:10:00.361897    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x" event={"ID":"66eb3ae5-5efe-4070-a999-d399e0a875f0","Type":"ContainerDied","Data":"b4dd36160c6884d19da84288cb730f2273d3b7ca0974d7d47296084893bc1373"}
Aug 05 18:10:00.362039 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:10:00.362023    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x" event={"ID":"66eb3ae5-5efe-4070-a999-d399e0a875f0","Type":"ContainerDied","Data":"d752966dc9429b0f28c07da9013ca129e3ec4068a383c1e9b4beeb018b549e28"}
Aug 05 18:10:00.362158 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:10:00.362053    2569 scope.go:117] "RemoveContainer" containerID="b4dd36160c6884d19da84288cb730f2273d3b7ca0974d7d47296084893bc1373"
Aug 05 18:10:00.372444 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:10:00.372430    2569 scope.go:117] "RemoveContainer" containerID="b4dd36160c6884d19da84288cb730f2273d3b7ca0974d7d47296084893bc1373"
Aug 05 18:10:00.372693 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:10:00.372676    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4dd36160c6884d19da84288cb730f2273d3b7ca0974d7d47296084893bc1373\": container with ID starting with b4dd36160c6884d19da84288cb730f2273d3b7ca0974d7d47296084893bc1373 not found: ID does not exist" containerID="b4dd36160c6884d19da84288cb730f2273d3b7ca0974d7d47296084893bc1373"
Aug 05 18:10:00.372736 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:10:00.372701    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4dd36160c6884d19da84288cb730f2273d3b7ca0974d7d47296084893bc1373"} err="failed to get container status \"b4dd36160c6884d19da84288cb730f2273d3b7ca0974d7d47296084893bc1373\": rpc error: code = NotFound desc = could not find container \"b4dd36160c6884d19da84288cb730f2273d3b7ca0974d7d47296084893bc1373\": container with ID starting with b4dd36160c6884d19da84288cb730f2273d3b7ca0974d7d47296084893bc1373 not found: ID does not exist"
Aug 05 18:10:00.386226 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:10:00.386205    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x"]
Aug 05 18:10:00.389813 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:10:00.389793    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bkf7x"]
Aug 05 18:10:01.150441 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:10:01.150407    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66eb3ae5-5efe-4070-a999-d399e0a875f0" path="/var/lib/kubelet/pods/66eb3ae5-5efe-4070-a999-d399e0a875f0/volumes"
Aug 05 18:10:12.369666 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:10:12.369621    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:11:08.372613 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:08.372579    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:11:08.472241 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:08.472172    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:11:12.282068 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:12.282029    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:11:12.382926 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:12.382893    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:11:15.166869 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.166837    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t7hxb"]
Aug 05 18:11:15.167279 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.167226    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80db473b-a120-4c89-8631-809cecd830b7" containerName="payload-processing"
Aug 05 18:11:15.167279 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.167239    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="80db473b-a120-4c89-8631-809cecd830b7" containerName="payload-processing"
Aug 05 18:11:15.167279 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.167247    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43768536-9b9f-4966-97ca-e8de5bbdb564" containerName="maas-api"
Aug 05 18:11:15.167279 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.167253    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="43768536-9b9f-4966-97ca-e8de5bbdb564" containerName="maas-api"
Aug 05 18:11:15.167279 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.167265    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66eb3ae5-5efe-4070-a999-d399e0a875f0" containerName="manager"
Aug 05 18:11:15.167279 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.167270    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="66eb3ae5-5efe-4070-a999-d399e0a875f0" containerName="manager"
Aug 05 18:11:15.167279 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.167278    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c582154-cd7d-4517-83b2-bec670979d41" containerName="payload-pre-processing"
Aug 05 18:11:15.167279 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.167283    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c582154-cd7d-4517-83b2-bec670979d41" containerName="payload-pre-processing"
Aug 05 18:11:15.167560 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.167353    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c582154-cd7d-4517-83b2-bec670979d41" containerName="payload-pre-processing"
Aug 05 18:11:15.167560 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.167362    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="43768536-9b9f-4966-97ca-e8de5bbdb564" containerName="maas-api"
Aug 05 18:11:15.167560 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.167368    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="66eb3ae5-5efe-4070-a999-d399e0a875f0" containerName="manager"
Aug 05 18:11:15.167560 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.167377    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="80db473b-a120-4c89-8631-809cecd830b7" containerName="payload-processing"
Aug 05 18:11:15.170421 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.170404    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t7hxb"
Aug 05 18:11:15.173697 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.173676    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hfkjp\""
Aug 05 18:11:15.194457 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.194431    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t7hxb"]
Aug 05 18:11:15.288458 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.288431    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkjlx\" (UniqueName: \"kubernetes.io/projected/48cf115d-5c47-4346-9d57-9a8117839e13-kube-api-access-xkjlx\") pod \"kuadrant-operator-controller-manager-68cf589589-t7hxb\" (UID: \"48cf115d-5c47-4346-9d57-9a8117839e13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t7hxb"
Aug 05 18:11:15.288599 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.288515    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48cf115d-5c47-4346-9d57-9a8117839e13-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-t7hxb\" (UID: \"48cf115d-5c47-4346-9d57-9a8117839e13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t7hxb"
Aug 05 18:11:15.389278 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.389252    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkjlx\" (UniqueName: \"kubernetes.io/projected/48cf115d-5c47-4346-9d57-9a8117839e13-kube-api-access-xkjlx\") pod \"kuadrant-operator-controller-manager-68cf589589-t7hxb\" (UID: \"48cf115d-5c47-4346-9d57-9a8117839e13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t7hxb"
Aug 05 18:11:15.389419 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.389324    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48cf115d-5c47-4346-9d57-9a8117839e13-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-t7hxb\" (UID: \"48cf115d-5c47-4346-9d57-9a8117839e13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t7hxb"
Aug 05 18:11:15.389691 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.389674    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48cf115d-5c47-4346-9d57-9a8117839e13-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-t7hxb\" (UID: \"48cf115d-5c47-4346-9d57-9a8117839e13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t7hxb"
Aug 05 18:11:15.400186 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.400155    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkjlx\" (UniqueName: \"kubernetes.io/projected/48cf115d-5c47-4346-9d57-9a8117839e13-kube-api-access-xkjlx\") pod \"kuadrant-operator-controller-manager-68cf589589-t7hxb\" (UID: \"48cf115d-5c47-4346-9d57-9a8117839e13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t7hxb"
Aug 05 18:11:15.481062 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.480999    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t7hxb"
Aug 05 18:11:15.628214 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.628168    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t7hxb"]
Aug 05 18:11:15.630001 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:11:15.629974    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48cf115d_5c47_4346_9d57_9a8117839e13.slice/crio-5903558a75b5af82b3b4f9efbd0390aa72b27c72dd439c8c110cda80de3b14bd WatchSource:0}: Error finding container 5903558a75b5af82b3b4f9efbd0390aa72b27c72dd439c8c110cda80de3b14bd: Status 404 returned error can't find the container with id 5903558a75b5af82b3b4f9efbd0390aa72b27c72dd439c8c110cda80de3b14bd
Aug 05 18:11:15.724437 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.724405    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t7hxb" event={"ID":"48cf115d-5c47-4346-9d57-9a8117839e13","Type":"ContainerStarted","Data":"717ccbcde434e4a89d9a48bbe4908a12dba7ea32b078be981cad9c9794f181bf"}
Aug 05 18:11:15.724551 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.724462    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t7hxb" event={"ID":"48cf115d-5c47-4346-9d57-9a8117839e13","Type":"ContainerStarted","Data":"5903558a75b5af82b3b4f9efbd0390aa72b27c72dd439c8c110cda80de3b14bd"}
Aug 05 18:11:15.724551 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.724523    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t7hxb"
Aug 05 18:11:15.750365 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:15.750278    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t7hxb" podStartSLOduration=0.750259262 podStartE2EDuration="750.259262ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:11:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:11:15.748133567 +0000 UTC m=+2283.100808152" watchObservedRunningTime="2026-08-05 18:11:15.750259262 +0000 UTC m=+2283.102933858"
Aug 05 18:11:26.731235 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:26.731174    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t7hxb"
Aug 05 18:11:33.196194 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:33.196150    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:11:33.507121 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:33.507031    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:11:33.608986 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:33.608947    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:11:36.274477 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:36.274434    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:11:36.366230 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:36.366194    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:11:37.397941 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:37.397910    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"]
Aug 05 18:11:37.402198 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:37.402164    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"
Aug 05 18:11:37.404708 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:37.404678    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9e3b0986-serving-cert\""
Aug 05 18:11:37.404876 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:37.404849    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jdhhr\""
Aug 05 18:11:37.417989 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:37.417961    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"]
Aug 05 18:11:37.455695 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:37.455663    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:11:37.486781 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:37.486747    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n2k4\" (UniqueName: \"kubernetes.io/projected/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-kube-api-access-7n2k4\") pod \"maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt\" (UID: \"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"
Aug 05 18:11:37.486978 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:37.486792    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-maas-api-tls\") pod \"maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt\" (UID: \"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"
Aug 05 18:11:37.486978 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:37.486846    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-openshift-service-ca\") pod \"maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt\" (UID: \"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"
Aug 05 18:11:37.587480 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:37.587449    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-maas-api-tls\") pod \"maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt\" (UID: \"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"
Aug 05 18:11:37.587663 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:37.587527    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-openshift-service-ca\") pod \"maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt\" (UID: \"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"
Aug 05 18:11:37.587663 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:11:37.587595    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-serving-cert: secret "maas-api-e2e-mt-9e3b0986-serving-cert" not found
Aug 05 18:11:37.587663 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:37.587632    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n2k4\" (UniqueName: \"kubernetes.io/projected/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-kube-api-access-7n2k4\") pod \"maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt\" (UID: \"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"
Aug 05 18:11:37.587830 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:11:37.587665    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-maas-api-tls podName:fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7 nodeName:}" failed. No retries permitted until 2026-08-05 18:11:38.08764313 +0000 UTC m=+2305.440317705 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-maas-api-tls") pod "maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt" (UID: "fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7") : secret "maas-api-e2e-mt-9e3b0986-serving-cert" not found
Aug 05 18:11:37.588345 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:37.588324    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-openshift-service-ca\") pod \"maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt\" (UID: \"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"
Aug 05 18:11:37.599310 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:37.599286    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n2k4\" (UniqueName: \"kubernetes.io/projected/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-kube-api-access-7n2k4\") pod \"maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt\" (UID: \"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"
Aug 05 18:11:37.676397 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:37.676308    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:11:37.970359 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:37.970273    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:11:38.078830 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.078796    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:11:38.093078 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.093050    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-maas-api-tls\") pod \"maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt\" (UID: \"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"
Aug 05 18:11:38.098239 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.096346    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-maas-api-tls\") pod \"maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt\" (UID: \"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"
Aug 05 18:11:38.109789 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.109759    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz"]
Aug 05 18:11:38.114534 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.114504    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz"]
Aug 05 18:11:38.114675 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.114625    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz"
Aug 05 18:11:38.115376 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.115349    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz"]
Aug 05 18:11:38.118889 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.118864    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-9e3b0986\""
Aug 05 18:11:38.119307 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.119287    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz"
Aug 05 18:11:38.119432 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.119348    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-9e3b0986-dockercfg-mbjtn\""
Aug 05 18:11:38.130482 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.130460    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz"]
Aug 05 18:11:38.206202 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.205696    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf8r4\" (UniqueName: \"kubernetes.io/projected/fe7cdddc-309d-4c2e-8def-dc026cb09404-kube-api-access-mf8r4\") pod \"payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz\" (UID: \"fe7cdddc-309d-4c2e-8def-dc026cb09404\") " pod="openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz"
Aug 05 18:11:38.206202 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.205749    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb0d63d3-84ef-4ba0-9e47-d881d8a341bd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz\" (UID: \"bb0d63d3-84ef-4ba0-9e47-d881d8a341bd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz"
Aug 05 18:11:38.206202 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.205784    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgnbc\" (UniqueName: \"kubernetes.io/projected/bb0d63d3-84ef-4ba0-9e47-d881d8a341bd-kube-api-access-fgnbc\") pod \"payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz\" (UID: \"bb0d63d3-84ef-4ba0-9e47-d881d8a341bd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz"
Aug 05 18:11:38.206202 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.205820    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe7cdddc-309d-4c2e-8def-dc026cb09404-plugins-config-volume\") pod \"payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz\" (UID: \"fe7cdddc-309d-4c2e-8def-dc026cb09404\") " pod="openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz"
Aug 05 18:11:38.307681 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.307591    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf8r4\" (UniqueName: \"kubernetes.io/projected/fe7cdddc-309d-4c2e-8def-dc026cb09404-kube-api-access-mf8r4\") pod \"payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz\" (UID: \"fe7cdddc-309d-4c2e-8def-dc026cb09404\") " pod="openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz"
Aug 05 18:11:38.307681 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.307648    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb0d63d3-84ef-4ba0-9e47-d881d8a341bd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz\" (UID: \"bb0d63d3-84ef-4ba0-9e47-d881d8a341bd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz"
Aug 05 18:11:38.307887 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.307696    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgnbc\" (UniqueName: \"kubernetes.io/projected/bb0d63d3-84ef-4ba0-9e47-d881d8a341bd-kube-api-access-fgnbc\") pod \"payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz\" (UID: \"bb0d63d3-84ef-4ba0-9e47-d881d8a341bd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz"
Aug 05 18:11:38.307887 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.307743    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe7cdddc-309d-4c2e-8def-dc026cb09404-plugins-config-volume\") pod \"payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz\" (UID: \"fe7cdddc-309d-4c2e-8def-dc026cb09404\") " pod="openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz"
Aug 05 18:11:38.308352 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.308329    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb0d63d3-84ef-4ba0-9e47-d881d8a341bd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz\" (UID: \"bb0d63d3-84ef-4ba0-9e47-d881d8a341bd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz"
Aug 05 18:11:38.308560 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.308537    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe7cdddc-309d-4c2e-8def-dc026cb09404-plugins-config-volume\") pod \"payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz\" (UID: \"fe7cdddc-309d-4c2e-8def-dc026cb09404\") " pod="openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz"
Aug 05 18:11:38.315312 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.315285    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"
Aug 05 18:11:38.325968 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.325940    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf8r4\" (UniqueName: \"kubernetes.io/projected/fe7cdddc-309d-4c2e-8def-dc026cb09404-kube-api-access-mf8r4\") pod \"payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz\" (UID: \"fe7cdddc-309d-4c2e-8def-dc026cb09404\") " pod="openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz"
Aug 05 18:11:38.326811 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.326783    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgnbc\" (UniqueName: \"kubernetes.io/projected/bb0d63d3-84ef-4ba0-9e47-d881d8a341bd-kube-api-access-fgnbc\") pod \"payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz\" (UID: \"bb0d63d3-84ef-4ba0-9e47-d881d8a341bd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz"
Aug 05 18:11:38.430396 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.430314    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz"
Aug 05 18:11:38.436820 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.436786    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz"
Aug 05 18:11:38.486888 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.486846    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"]
Aug 05 18:11:38.488466 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:11:38.488429    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd8c8f6e_5402_41ac_95c3_e4093ef4d2b7.slice/crio-d7e37836d5b688693b1c2d288a2b8b2c6dd0636fc21f5efc631732735c2cc52c WatchSource:0}: Error finding container d7e37836d5b688693b1c2d288a2b8b2c6dd0636fc21f5efc631732735c2cc52c: Status 404 returned error can't find the container with id d7e37836d5b688693b1c2d288a2b8b2c6dd0636fc21f5efc631732735c2cc52c
Aug 05 18:11:38.489873 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.489855    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 18:11:38.613234 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.613137    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz"]
Aug 05 18:11:38.617620 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:11:38.617564    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe7cdddc_309d_4c2e_8def_dc026cb09404.slice/crio-bcdf6c36395835c133f14c7955bf7d8773d3c040610cc18732c9190e435535dc WatchSource:0}: Error finding container bcdf6c36395835c133f14c7955bf7d8773d3c040610cc18732c9190e435535dc: Status 404 returned error can't find the container with id bcdf6c36395835c133f14c7955bf7d8773d3c040610cc18732c9190e435535dc
Aug 05 18:11:38.647828 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.647799    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz"]
Aug 05 18:11:38.658488 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:11:38.658456    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb0d63d3_84ef_4ba0_9e47_d881d8a341bd.slice/crio-bddc7492d493ee802c06cbbcca1aed934f494d506ce9c7aa418b9fb24a2e3bf4 WatchSource:0}: Error finding container bddc7492d493ee802c06cbbcca1aed934f494d506ce9c7aa418b9fb24a2e3bf4: Status 404 returned error can't find the container with id bddc7492d493ee802c06cbbcca1aed934f494d506ce9c7aa418b9fb24a2e3bf4
Aug 05 18:11:38.848003 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.847916    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz" event={"ID":"fe7cdddc-309d-4c2e-8def-dc026cb09404","Type":"ContainerStarted","Data":"1dd6b0aa612e6e4b5a001c8500624b81f56aa098835dab6f402bc93a42b593f4"}
Aug 05 18:11:38.848003 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.847980    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz" event={"ID":"fe7cdddc-309d-4c2e-8def-dc026cb09404","Type":"ContainerStarted","Data":"bcdf6c36395835c133f14c7955bf7d8773d3c040610cc18732c9190e435535dc"}
Aug 05 18:11:38.848720 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.848700    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz"
Aug 05 18:11:38.850156 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.850125    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt" event={"ID":"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7","Type":"ContainerStarted","Data":"d7e37836d5b688693b1c2d288a2b8b2c6dd0636fc21f5efc631732735c2cc52c"}
Aug 05 18:11:38.852075 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.852045    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz" event={"ID":"bb0d63d3-84ef-4ba0-9e47-d881d8a341bd","Type":"ContainerStarted","Data":"0e6f5a27ca34e360ce91918f47865753bae25f4c4a8dec25da5a651a44513bd1"}
Aug 05 18:11:38.852216 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.852104    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz"
Aug 05 18:11:38.852216 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.852130    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz" event={"ID":"bb0d63d3-84ef-4ba0-9e47-d881d8a341bd","Type":"ContainerStarted","Data":"bddc7492d493ee802c06cbbcca1aed934f494d506ce9c7aa418b9fb24a2e3bf4"}
Aug 05 18:11:38.869348 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.869306    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz" podStartSLOduration=0.869293203 podStartE2EDuration="869.293203ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:11:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:11:38.86706608 +0000 UTC m=+2306.219740676" watchObservedRunningTime="2026-08-05 18:11:38.869293203 +0000 UTC m=+2306.221967828"
Aug 05 18:11:38.892459 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:38.892420    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz" podStartSLOduration=0.892406472 podStartE2EDuration="892.406472ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:11:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:11:38.890667461 +0000 UTC m=+2306.243342050" watchObservedRunningTime="2026-08-05 18:11:38.892406472 +0000 UTC m=+2306.245081109"
Aug 05 18:11:39.858854 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:39.858812    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt" event={"ID":"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7","Type":"ContainerStarted","Data":"90d99979edc29b698bf5f787a9c990d1282725f51ec3fe93b581e5bac9ddca07"}
Aug 05 18:11:39.859460 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:39.859415    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"
Aug 05 18:11:39.879499 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:39.879454    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt" podStartSLOduration=2.398305 podStartE2EDuration="2.879440168s" totalImagesPullingTime="481.135168ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:11:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:11:39.877761004 +0000 UTC m=+2307.230435598" watchObservedRunningTime="2026-08-05 18:11:39.879440168 +0000 UTC m=+2307.232114762"
Aug 05 18:11:46.877369 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:46.877333    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"
Aug 05 18:11:49.861010 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:49.860970    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz"
Aug 05 18:11:50.867633 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:11:50.867604    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz"
Aug 05 18:12:01.232809 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:01.232777    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:12:07.276469 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:07.276387    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:12:50.917452 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:50.917412    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"]
Aug 05 18:12:50.917990 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:50.917652    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt" podUID="fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7" containerName="maas-api" containerID="cri-o://90d99979edc29b698bf5f787a9c990d1282725f51ec3fe93b581e5bac9ddca07" gracePeriod=30
Aug 05 18:12:50.947088 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:50.947061    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:12:50.971077 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:50.971045    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz"]
Aug 05 18:12:50.971386 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:50.971329    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz" podUID="fe7cdddc-309d-4c2e-8def-dc026cb09404" containerName="payload-processing" containerID="cri-o://1dd6b0aa612e6e4b5a001c8500624b81f56aa098835dab6f402bc93a42b593f4" gracePeriod=30
Aug 05 18:12:50.979841 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:50.979805    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz"]
Aug 05 18:12:50.980231 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:50.980200    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz" podUID="bb0d63d3-84ef-4ba0-9e47-d881d8a341bd" containerName="payload-pre-processing" containerID="cri-o://0e6f5a27ca34e360ce91918f47865753bae25f4c4a8dec25da5a651a44513bd1" gracePeriod=30
Aug 05 18:12:51.016526 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.016496    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:12:51.097719 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.097684    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:12:51.203924 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.203895    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:12:51.239787 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.239752    2569 generic.go:356] "Generic (PLEG): container finished" podID="fe7cdddc-309d-4c2e-8def-dc026cb09404" containerID="1dd6b0aa612e6e4b5a001c8500624b81f56aa098835dab6f402bc93a42b593f4" exitCode=0
Aug 05 18:12:51.239942 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.239826    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz" event={"ID":"fe7cdddc-309d-4c2e-8def-dc026cb09404","Type":"ContainerDied","Data":"1dd6b0aa612e6e4b5a001c8500624b81f56aa098835dab6f402bc93a42b593f4"}
Aug 05 18:12:51.241850 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.241824    2569 generic.go:356] "Generic (PLEG): container finished" podID="fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7" containerID="90d99979edc29b698bf5f787a9c990d1282725f51ec3fe93b581e5bac9ddca07" exitCode=0
Aug 05 18:12:51.241988 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.241930    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt" event={"ID":"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7","Type":"ContainerDied","Data":"90d99979edc29b698bf5f787a9c990d1282725f51ec3fe93b581e5bac9ddca07"}
Aug 05 18:12:51.244214 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.244153    2569 generic.go:356] "Generic (PLEG): container finished" podID="bb0d63d3-84ef-4ba0-9e47-d881d8a341bd" containerID="0e6f5a27ca34e360ce91918f47865753bae25f4c4a8dec25da5a651a44513bd1" exitCode=0
Aug 05 18:12:51.244493 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.244231    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz" event={"ID":"bb0d63d3-84ef-4ba0-9e47-d881d8a341bd","Type":"ContainerDied","Data":"0e6f5a27ca34e360ce91918f47865753bae25f4c4a8dec25da5a651a44513bd1"}
Aug 05 18:12:51.268558 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.268532    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz"
Aug 05 18:12:51.271662 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.271640    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"
Aug 05 18:12:51.293330 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.293306    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz"
Aug 05 18:12:51.397523 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.397133    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7n2k4\" (UniqueName: \"kubernetes.io/projected/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-kube-api-access-7n2k4\") pod \"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7\" (UID: \"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7\") "
Aug 05 18:12:51.397523 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.397282    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb0d63d3-84ef-4ba0-9e47-d881d8a341bd-plugins-config-volume\") pod \"bb0d63d3-84ef-4ba0-9e47-d881d8a341bd\" (UID: \"bb0d63d3-84ef-4ba0-9e47-d881d8a341bd\") "
Aug 05 18:12:51.397806 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.397762    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe7cdddc-309d-4c2e-8def-dc026cb09404-plugins-config-volume\") pod \"fe7cdddc-309d-4c2e-8def-dc026cb09404\" (UID: \"fe7cdddc-309d-4c2e-8def-dc026cb09404\") "
Aug 05 18:12:51.397882 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.397834    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgnbc\" (UniqueName: \"kubernetes.io/projected/bb0d63d3-84ef-4ba0-9e47-d881d8a341bd-kube-api-access-fgnbc\") pod \"bb0d63d3-84ef-4ba0-9e47-d881d8a341bd\" (UID: \"bb0d63d3-84ef-4ba0-9e47-d881d8a341bd\") "
Aug 05 18:12:51.397937 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.397919    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-maas-api-tls\") pod \"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7\" (UID: \"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7\") "
Aug 05 18:12:51.397997 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.397969    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-openshift-service-ca\") pod \"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7\" (UID: \"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7\") "
Aug 05 18:12:51.398049 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.398009    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mf8r4\" (UniqueName: \"kubernetes.io/projected/fe7cdddc-309d-4c2e-8def-dc026cb09404-kube-api-access-mf8r4\") pod \"fe7cdddc-309d-4c2e-8def-dc026cb09404\" (UID: \"fe7cdddc-309d-4c2e-8def-dc026cb09404\") "
Aug 05 18:12:51.398244 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.398205    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb0d63d3-84ef-4ba0-9e47-d881d8a341bd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bb0d63d3-84ef-4ba0-9e47-d881d8a341bd" (UID: "bb0d63d3-84ef-4ba0-9e47-d881d8a341bd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:12:51.398717 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.398412    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb0d63d3-84ef-4ba0-9e47-d881d8a341bd-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:12:51.398717 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.398640    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe7cdddc-309d-4c2e-8def-dc026cb09404-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fe7cdddc-309d-4c2e-8def-dc026cb09404" (UID: "fe7cdddc-309d-4c2e-8def-dc026cb09404"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:12:51.399490 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.399210    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7" (UID: "fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:12:51.401130 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.401106    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb0d63d3-84ef-4ba0-9e47-d881d8a341bd-kube-api-access-fgnbc" (OuterVolumeSpecName: "kube-api-access-fgnbc") pod "bb0d63d3-84ef-4ba0-9e47-d881d8a341bd" (UID: "bb0d63d3-84ef-4ba0-9e47-d881d8a341bd"). InnerVolumeSpecName "kube-api-access-fgnbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:12:51.401344 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.401325    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-kube-api-access-7n2k4" (OuterVolumeSpecName: "kube-api-access-7n2k4") pod "fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7" (UID: "fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7"). InnerVolumeSpecName "kube-api-access-7n2k4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:12:51.401443 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.401413    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7" (UID: "fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:12:51.401775 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.401738    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe7cdddc-309d-4c2e-8def-dc026cb09404-kube-api-access-mf8r4" (OuterVolumeSpecName: "kube-api-access-mf8r4") pod "fe7cdddc-309d-4c2e-8def-dc026cb09404" (UID: "fe7cdddc-309d-4c2e-8def-dc026cb09404"). InnerVolumeSpecName "kube-api-access-mf8r4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:12:51.499432 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.499351    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-maas-api-tls\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:12:51.499432 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.499379    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-openshift-service-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:12:51.499432 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.499389    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mf8r4\" (UniqueName: \"kubernetes.io/projected/fe7cdddc-309d-4c2e-8def-dc026cb09404-kube-api-access-mf8r4\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:12:51.499432 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.499397    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7n2k4\" (UniqueName: \"kubernetes.io/projected/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7-kube-api-access-7n2k4\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:12:51.499432 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.499406    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe7cdddc-309d-4c2e-8def-dc026cb09404-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:12:51.499432 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:51.499415    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgnbc\" (UniqueName: \"kubernetes.io/projected/bb0d63d3-84ef-4ba0-9e47-d881d8a341bd-kube-api-access-fgnbc\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:12:52.251136 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:52.251049    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz" event={"ID":"fe7cdddc-309d-4c2e-8def-dc026cb09404","Type":"ContainerDied","Data":"bcdf6c36395835c133f14c7955bf7d8773d3c040610cc18732c9190e435535dc"}
Aug 05 18:12:52.251136 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:52.251116    2569 scope.go:117] "RemoveContainer" containerID="1dd6b0aa612e6e4b5a001c8500624b81f56aa098835dab6f402bc93a42b593f4"
Aug 05 18:12:52.251653 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:52.251058    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz"
Aug 05 18:12:52.252531 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:52.252515    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"
Aug 05 18:12:52.252635 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:52.252533    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt" event={"ID":"fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7","Type":"ContainerDied","Data":"d7e37836d5b688693b1c2d288a2b8b2c6dd0636fc21f5efc631732735c2cc52c"}
Aug 05 18:12:52.255331 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:52.255304    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz" event={"ID":"bb0d63d3-84ef-4ba0-9e47-d881d8a341bd","Type":"ContainerDied","Data":"bddc7492d493ee802c06cbbcca1aed934f494d506ce9c7aa418b9fb24a2e3bf4"}
Aug 05 18:12:52.255491 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:52.255395    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz"
Aug 05 18:12:52.264597 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:52.263851    2569 scope.go:117] "RemoveContainer" containerID="90d99979edc29b698bf5f787a9c990d1282725f51ec3fe93b581e5bac9ddca07"
Aug 05 18:12:52.275355 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:52.275329    2569 scope.go:117] "RemoveContainer" containerID="0e6f5a27ca34e360ce91918f47865753bae25f4c4a8dec25da5a651a44513bd1"
Aug 05 18:12:52.290665 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:52.290640    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz"]
Aug 05 18:12:52.292700 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:52.292676    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9e3b0986-8694c4bdc4-8s4kz"]
Aug 05 18:12:52.302470 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:52.302449    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"]
Aug 05 18:12:52.305968 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:52.305944    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9e3b0986-7c778cbc5b-khnvt"]
Aug 05 18:12:52.317037 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:52.317018    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz"]
Aug 05 18:12:52.321761 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:52.321740    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9e3b0986-5b844f7cb-d5wrz"]
Aug 05 18:12:53.151620 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:53.151588    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb0d63d3-84ef-4ba0-9e47-d881d8a341bd" path="/var/lib/kubelet/pods/bb0d63d3-84ef-4ba0-9e47-d881d8a341bd/volumes"
Aug 05 18:12:53.151995 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:53.151982    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7" path="/var/lib/kubelet/pods/fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7/volumes"
Aug 05 18:12:53.152310 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:53.152297    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe7cdddc-309d-4c2e-8def-dc026cb09404" path="/var/lib/kubelet/pods/fe7cdddc-309d-4c2e-8def-dc026cb09404/volumes"
Aug 05 18:12:54.300966 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:54.300933    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:12:59.370383 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:12:59.370349    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:13:36.173315 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:13:36.173239    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:13:36.924372 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:13:36.924336    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:13:42.865133 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:13:42.865096    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:13:43.006803 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:13:43.006765    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:13:43.130142 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:13:43.130060    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:13:43.354718 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:13:43.354683    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:13:54.690311 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:13:54.690274    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:13:54.784418 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:13:54.784385    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:13:55.456822 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:13:55.456785    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:13:55.554148 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:13:55.554115    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:13:55.651785 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:13:55.651749    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:13:56.851481 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:13:56.851445    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:03.941792 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:03.941757    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:04.042500 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:04.042469    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:04.608310 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:04.608269    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:11.844378 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:11.844333    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:11.985132 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:11.985097    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:12.116888 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:12.116806    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:22.543784 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:22.543749    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:22.770670 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:22.770629    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:26.222803 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:26.222768    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:43.713433 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:43.713395    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:50.286053 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:50.286016    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:50.377392 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:50.377356    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:50.727393 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:50.727363    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:51.053856 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:51.053770    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:51.405652 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:51.405621    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:54.109861 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:54.109831    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:54.209897 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:54.209870    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:55.312949 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.312920    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"]
Aug 05 18:14:55.313365 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.313352    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe7cdddc-309d-4c2e-8def-dc026cb09404" containerName="payload-processing"
Aug 05 18:14:55.313423 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.313367    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe7cdddc-309d-4c2e-8def-dc026cb09404" containerName="payload-processing"
Aug 05 18:14:55.313423 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.313379    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb0d63d3-84ef-4ba0-9e47-d881d8a341bd" containerName="payload-pre-processing"
Aug 05 18:14:55.313423 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.313384    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb0d63d3-84ef-4ba0-9e47-d881d8a341bd" containerName="payload-pre-processing"
Aug 05 18:14:55.313423 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.313404    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7" containerName="maas-api"
Aug 05 18:14:55.313423 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.313410    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7" containerName="maas-api"
Aug 05 18:14:55.313580 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.313488    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb0d63d3-84ef-4ba0-9e47-d881d8a341bd" containerName="payload-pre-processing"
Aug 05 18:14:55.313580 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.313496    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe7cdddc-309d-4c2e-8def-dc026cb09404" containerName="payload-processing"
Aug 05 18:14:55.313580 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.313502    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd8c8f6e-5402-41ac-95c3-e4093ef4d2b7" containerName="maas-api"
Aug 05 18:14:55.316641 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.316626    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"
Aug 05 18:14:55.320409 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.320390    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jdhhr\""
Aug 05 18:14:55.320536 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.320502    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-9261d9-serving-cert\""
Aug 05 18:14:55.329030 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.329008    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"]
Aug 05 18:14:55.382101 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.382066    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:55.411631 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.411608    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20eaba87-48bd-4ee0-8b49-79aa2f0e541a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4\" (UID: \"20eaba87-48bd-4ee0-8b49-79aa2f0e541a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"
Aug 05 18:14:55.411767 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.411657    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mhbt\" (UniqueName: \"kubernetes.io/projected/20eaba87-48bd-4ee0-8b49-79aa2f0e541a-kube-api-access-7mhbt\") pod \"maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4\" (UID: \"20eaba87-48bd-4ee0-8b49-79aa2f0e541a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"
Aug 05 18:14:55.411767 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.411678    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20eaba87-48bd-4ee0-8b49-79aa2f0e541a-maas-api-tls\") pod \"maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4\" (UID: \"20eaba87-48bd-4ee0-8b49-79aa2f0e541a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"
Aug 05 18:14:55.473320 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.473298    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:55.512530 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.512511    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20eaba87-48bd-4ee0-8b49-79aa2f0e541a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4\" (UID: \"20eaba87-48bd-4ee0-8b49-79aa2f0e541a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"
Aug 05 18:14:55.512635 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.512572    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mhbt\" (UniqueName: \"kubernetes.io/projected/20eaba87-48bd-4ee0-8b49-79aa2f0e541a-kube-api-access-7mhbt\") pod \"maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4\" (UID: \"20eaba87-48bd-4ee0-8b49-79aa2f0e541a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"
Aug 05 18:14:55.512635 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.512593    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20eaba87-48bd-4ee0-8b49-79aa2f0e541a-maas-api-tls\") pod \"maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4\" (UID: \"20eaba87-48bd-4ee0-8b49-79aa2f0e541a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"
Aug 05 18:14:55.513132 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.513115    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20eaba87-48bd-4ee0-8b49-79aa2f0e541a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4\" (UID: \"20eaba87-48bd-4ee0-8b49-79aa2f0e541a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"
Aug 05 18:14:55.515071 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.515052    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20eaba87-48bd-4ee0-8b49-79aa2f0e541a-maas-api-tls\") pod \"maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4\" (UID: \"20eaba87-48bd-4ee0-8b49-79aa2f0e541a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"
Aug 05 18:14:55.525918 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.525893    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mhbt\" (UniqueName: \"kubernetes.io/projected/20eaba87-48bd-4ee0-8b49-79aa2f0e541a-kube-api-access-7mhbt\") pod \"maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4\" (UID: \"20eaba87-48bd-4ee0-8b49-79aa2f0e541a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"
Aug 05 18:14:55.628833 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.628817    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"
Aug 05 18:14:55.769145 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.769120    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"]
Aug 05 18:14:55.771172 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:14:55.771143    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20eaba87_48bd_4ee0_8b49_79aa2f0e541a.slice/crio-84c52b53be053c5800ded5bc24f0d3eb836004e4e216a2760b503ce820d46f69 WatchSource:0}: Error finding container 84c52b53be053c5800ded5bc24f0d3eb836004e4e216a2760b503ce820d46f69: Status 404 returned error can't find the container with id 84c52b53be053c5800ded5bc24f0d3eb836004e4e216a2760b503ce820d46f69
Aug 05 18:14:55.895877 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.895801    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4" event={"ID":"20eaba87-48bd-4ee0-8b49-79aa2f0e541a","Type":"ContainerStarted","Data":"84c52b53be053c5800ded5bc24f0d3eb836004e4e216a2760b503ce820d46f69"}
Aug 05 18:14:55.936926 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:55.936894    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:56.027388 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.027358    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd"]
Aug 05 18:14:56.028419 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.028384    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:56.032557 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.032542    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd"
Aug 05 18:14:56.035068 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.035047    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-9261d9\""
Aug 05 18:14:56.035217 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.035106    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-9261d9-dockercfg-g9fwq\""
Aug 05 18:14:56.042852 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.042795    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd"]
Aug 05 18:14:56.068705 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.068680    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt"]
Aug 05 18:14:56.072434 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.072416    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt"
Aug 05 18:14:56.073505 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.073483    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt"]
Aug 05 18:14:56.118514 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.118482    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27c11d69-c064-49fb-8f70-758da8599957-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt\" (UID: \"27c11d69-c064-49fb-8f70-758da8599957\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt"
Aug 05 18:14:56.118632 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.118523    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjzkq\" (UniqueName: \"kubernetes.io/projected/27c11d69-c064-49fb-8f70-758da8599957-kube-api-access-hjzkq\") pod \"payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt\" (UID: \"27c11d69-c064-49fb-8f70-758da8599957\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt"
Aug 05 18:14:56.118632 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.118569    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0776738a-0e68-4351-8cf1-9ba2e0ca2500-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd\" (UID: \"0776738a-0e68-4351-8cf1-9ba2e0ca2500\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd"
Aug 05 18:14:56.118632 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.118617    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x855n\" (UniqueName: \"kubernetes.io/projected/0776738a-0e68-4351-8cf1-9ba2e0ca2500-kube-api-access-x855n\") pod \"payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd\" (UID: \"0776738a-0e68-4351-8cf1-9ba2e0ca2500\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd"
Aug 05 18:14:56.219198 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.219114    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x855n\" (UniqueName: \"kubernetes.io/projected/0776738a-0e68-4351-8cf1-9ba2e0ca2500-kube-api-access-x855n\") pod \"payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd\" (UID: \"0776738a-0e68-4351-8cf1-9ba2e0ca2500\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd"
Aug 05 18:14:56.219322 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.219287    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27c11d69-c064-49fb-8f70-758da8599957-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt\" (UID: \"27c11d69-c064-49fb-8f70-758da8599957\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt"
Aug 05 18:14:56.219322 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.219316    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjzkq\" (UniqueName: \"kubernetes.io/projected/27c11d69-c064-49fb-8f70-758da8599957-kube-api-access-hjzkq\") pod \"payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt\" (UID: \"27c11d69-c064-49fb-8f70-758da8599957\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt"
Aug 05 18:14:56.219513 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.219484    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0776738a-0e68-4351-8cf1-9ba2e0ca2500-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd\" (UID: \"0776738a-0e68-4351-8cf1-9ba2e0ca2500\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd"
Aug 05 18:14:56.220035 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.220015    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27c11d69-c064-49fb-8f70-758da8599957-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt\" (UID: \"27c11d69-c064-49fb-8f70-758da8599957\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt"
Aug 05 18:14:56.220115 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.220084    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0776738a-0e68-4351-8cf1-9ba2e0ca2500-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd\" (UID: \"0776738a-0e68-4351-8cf1-9ba2e0ca2500\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd"
Aug 05 18:14:56.228046 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.228026    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x855n\" (UniqueName: \"kubernetes.io/projected/0776738a-0e68-4351-8cf1-9ba2e0ca2500-kube-api-access-x855n\") pod \"payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd\" (UID: \"0776738a-0e68-4351-8cf1-9ba2e0ca2500\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd"
Aug 05 18:14:56.228309 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.228292    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjzkq\" (UniqueName: \"kubernetes.io/projected/27c11d69-c064-49fb-8f70-758da8599957-kube-api-access-hjzkq\") pod \"payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt\" (UID: \"27c11d69-c064-49fb-8f70-758da8599957\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt"
Aug 05 18:14:56.357822 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.357797    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd"
Aug 05 18:14:56.383577 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.383554    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt"
Aug 05 18:14:56.514075 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.513528    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd"]
Aug 05 18:14:56.517043 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:14:56.517012    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0776738a_0e68_4351_8cf1_9ba2e0ca2500.slice/crio-74203e89efcfba0bc022abf79564fef3f11e21efc738c17ad5b5267e854e223d WatchSource:0}: Error finding container 74203e89efcfba0bc022abf79564fef3f11e21efc738c17ad5b5267e854e223d: Status 404 returned error can't find the container with id 74203e89efcfba0bc022abf79564fef3f11e21efc738c17ad5b5267e854e223d
Aug 05 18:14:56.538155 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.538121    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt"]
Aug 05 18:14:56.540025 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:14:56.540001    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c11d69_c064_49fb_8f70_758da8599957.slice/crio-3670404e13df5029201eed52064a5170c5185815ea1f30f146cd7cf4a789546e WatchSource:0}: Error finding container 3670404e13df5029201eed52064a5170c5185815ea1f30f146cd7cf4a789546e: Status 404 returned error can't find the container with id 3670404e13df5029201eed52064a5170c5185815ea1f30f146cd7cf4a789546e
Aug 05 18:14:56.902959 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.902925    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt" event={"ID":"27c11d69-c064-49fb-8f70-758da8599957","Type":"ContainerStarted","Data":"cb3541f3bf59ccfc7a8445116c6e5fb9590d6c042db3e4113ced8be954be82e3"}
Aug 05 18:14:56.903160 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.902979    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt" event={"ID":"27c11d69-c064-49fb-8f70-758da8599957","Type":"ContainerStarted","Data":"3670404e13df5029201eed52064a5170c5185815ea1f30f146cd7cf4a789546e"}
Aug 05 18:14:56.903160 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.903120    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt"
Aug 05 18:14:56.904564 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.904511    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd" event={"ID":"0776738a-0e68-4351-8cf1-9ba2e0ca2500","Type":"ContainerStarted","Data":"fb8907cd40afda25565b6a15c2a631dc605d68b83fd2215a36d27de650ec58a9"}
Aug 05 18:14:56.904564 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.904559    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd" event={"ID":"0776738a-0e68-4351-8cf1-9ba2e0ca2500","Type":"ContainerStarted","Data":"74203e89efcfba0bc022abf79564fef3f11e21efc738c17ad5b5267e854e223d"}
Aug 05 18:14:56.904739 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.904580    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd"
Aug 05 18:14:56.905988 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.905960    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4" event={"ID":"20eaba87-48bd-4ee0-8b49-79aa2f0e541a","Type":"ContainerStarted","Data":"d2f6680f9f005b0e5697d27312537102c5fe4afbb1870fcef99267b04e000d59"}
Aug 05 18:14:56.906115 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.906098    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"
Aug 05 18:14:56.921074 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.921027    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt" podStartSLOduration=0.921011911 podStartE2EDuration="921.011911ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:14:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:14:56.920928004 +0000 UTC m=+2504.273602613" watchObservedRunningTime="2026-08-05 18:14:56.921011911 +0000 UTC m=+2504.273686508"
Aug 05 18:14:56.940656 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.940608    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd" podStartSLOduration=0.940590159 podStartE2EDuration="940.590159ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:14:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:14:56.938336442 +0000 UTC m=+2504.291011036" watchObservedRunningTime="2026-08-05 18:14:56.940590159 +0000 UTC m=+2504.293264755"
Aug 05 18:14:56.964008 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:14:56.963956    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4" podStartSLOduration=1.467549475 podStartE2EDuration="1.963936105s" totalImagesPullingTime="496.38663ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:14:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:14:56.960892759 +0000 UTC m=+2504.313567353" watchObservedRunningTime="2026-08-05 18:14:56.963936105 +0000 UTC m=+2504.316610705"
Aug 05 18:15:02.916660 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:02.916634    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"
Aug 05 18:15:07.913420 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:07.913332    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt"
Aug 05 18:15:07.913420 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:07.913419    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd"
Aug 05 18:15:24.197062 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:24.197024    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:15:24.536441 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:24.536361    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:15:24.889862 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:24.889829    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:15:25.064238 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:25.064211    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:15:28.249039 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:28.248995    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:15:28.343247 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:28.343213    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:15:29.376198 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:29.376131    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:15:29.667874 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:29.667796    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:15:29.893417 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:29.893381    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:15:29.988097 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:29.988014    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:15:30.060640 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:30.060594    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv"]
Aug 05 18:15:30.065151 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:30.065125    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv"
Aug 05 18:15:30.067820 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:30.067796    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-955c47\""
Aug 05 18:15:30.067960 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:30.067893    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-955c47-dockercfg-5vm5z\""
Aug 05 18:15:30.072257 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:30.072230    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv"]
Aug 05 18:15:30.223416 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:30.223376    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blv5r\" (UniqueName: \"kubernetes.io/projected/c111cc1d-00dc-4d8c-8fc4-09167a862dd3-kube-api-access-blv5r\") pod \"payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv\" (UID: \"c111cc1d-00dc-4d8c-8fc4-09167a862dd3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv"
Aug 05 18:15:30.223416 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:30.223423    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c111cc1d-00dc-4d8c-8fc4-09167a862dd3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv\" (UID: \"c111cc1d-00dc-4d8c-8fc4-09167a862dd3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv"
Aug 05 18:15:30.324515 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:30.324487    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blv5r\" (UniqueName: \"kubernetes.io/projected/c111cc1d-00dc-4d8c-8fc4-09167a862dd3-kube-api-access-blv5r\") pod \"payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv\" (UID: \"c111cc1d-00dc-4d8c-8fc4-09167a862dd3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv"
Aug 05 18:15:30.324674 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:30.324533    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c111cc1d-00dc-4d8c-8fc4-09167a862dd3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv\" (UID: \"c111cc1d-00dc-4d8c-8fc4-09167a862dd3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv"
Aug 05 18:15:30.325263 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:30.325244    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c111cc1d-00dc-4d8c-8fc4-09167a862dd3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv\" (UID: \"c111cc1d-00dc-4d8c-8fc4-09167a862dd3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv"
Aug 05 18:15:30.333142 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:30.333114    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blv5r\" (UniqueName: \"kubernetes.io/projected/c111cc1d-00dc-4d8c-8fc4-09167a862dd3-kube-api-access-blv5r\") pod \"payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv\" (UID: \"c111cc1d-00dc-4d8c-8fc4-09167a862dd3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv"
Aug 05 18:15:30.378108 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:30.378086    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv"
Aug 05 18:15:30.522486 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:30.522458    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv"]
Aug 05 18:15:30.525350 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:15:30.525321    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc111cc1d_00dc_4d8c_8fc4_09167a862dd3.slice/crio-a46175e01c9b6c7435f58a1bedd1491784f043ab404f0737db65e5fae185cecf WatchSource:0}: Error finding container a46175e01c9b6c7435f58a1bedd1491784f043ab404f0737db65e5fae185cecf: Status 404 returned error can't find the container with id a46175e01c9b6c7435f58a1bedd1491784f043ab404f0737db65e5fae185cecf
Aug 05 18:15:31.081914 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:31.081878    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv" event={"ID":"c111cc1d-00dc-4d8c-8fc4-09167a862dd3","Type":"ContainerStarted","Data":"0c3f485b7f72c19cbf3e75ec2e00ff226500d1e08e3bc42796031b09f08d7f01"}
Aug 05 18:15:31.082096 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:31.081934    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv" event={"ID":"c111cc1d-00dc-4d8c-8fc4-09167a862dd3","Type":"ContainerStarted","Data":"a46175e01c9b6c7435f58a1bedd1491784f043ab404f0737db65e5fae185cecf"}
Aug 05 18:15:31.082096 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:31.081954    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv"
Aug 05 18:15:31.101509 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:31.101465    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv" podStartSLOduration=1.101449801 podStartE2EDuration="1.101449801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:15:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:15:31.099324846 +0000 UTC m=+2538.451999439" watchObservedRunningTime="2026-08-05 18:15:31.101449801 +0000 UTC m=+2538.454124395"
Aug 05 18:15:35.558159 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:35.558123    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:15:42.088348 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:42.088316    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv"
Aug 05 18:15:53.241851 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:15:53.241814    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:16:06.327924 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.327890    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"]
Aug 05 18:16:06.331438 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.331416    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.335718 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.335697    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-9261d9\"/\"kube-root-ca.crt\""
Aug 05 18:16:06.336854 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.336838    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-9261d9\"/\"openshift-service-ca.crt\""
Aug 05 18:16:06.337398 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.337203    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-9261d9\"/\"test-model-9261d9-kserve-self-signed-certs\""
Aug 05 18:16:06.337398 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.337215    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-9261d9\"/\"default-dockercfg-p2shb\""
Aug 05 18:16:06.343076 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.343056    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"]
Aug 05 18:16:06.363800 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.363770    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:16:06.450740 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.450709    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-dshm\") pod \"test-model-9261d9-kserve-7bc497b87f-wcvlx\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") " pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.450740 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.450739    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-model-cache\") pod \"test-model-9261d9-kserve-7bc497b87f-wcvlx\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") " pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.450942 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.450768    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-home\") pod \"test-model-9261d9-kserve-7bc497b87f-wcvlx\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") " pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.450942 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.450824    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-tmp-dir\") pod \"test-model-9261d9-kserve-7bc497b87f-wcvlx\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") " pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.450942 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.450853    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-tls-certs\") pod \"test-model-9261d9-kserve-7bc497b87f-wcvlx\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") " pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.451097 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.450930    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z9zd\" (UniqueName: \"kubernetes.io/projected/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-kube-api-access-5z9zd\") pod \"test-model-9261d9-kserve-7bc497b87f-wcvlx\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") " pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.551929 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.551899    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z9zd\" (UniqueName: \"kubernetes.io/projected/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-kube-api-access-5z9zd\") pod \"test-model-9261d9-kserve-7bc497b87f-wcvlx\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") " pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.552078 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.551944    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-dshm\") pod \"test-model-9261d9-kserve-7bc497b87f-wcvlx\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") " pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.552078 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.551964    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-model-cache\") pod \"test-model-9261d9-kserve-7bc497b87f-wcvlx\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") " pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.552078 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.551997    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-home\") pod \"test-model-9261d9-kserve-7bc497b87f-wcvlx\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") " pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.552078 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.552019    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-tmp-dir\") pod \"test-model-9261d9-kserve-7bc497b87f-wcvlx\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") " pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.552078 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.552064    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-tls-certs\") pod \"test-model-9261d9-kserve-7bc497b87f-wcvlx\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") " pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.552464 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.552444    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-model-cache\") pod \"test-model-9261d9-kserve-7bc497b87f-wcvlx\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") " pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.552552 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.552527    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-tmp-dir\") pod \"test-model-9261d9-kserve-7bc497b87f-wcvlx\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") " pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.552700 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.552682    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-home\") pod \"test-model-9261d9-kserve-7bc497b87f-wcvlx\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") " pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.554446 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.554427    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-dshm\") pod \"test-model-9261d9-kserve-7bc497b87f-wcvlx\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") " pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.554712 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.554695    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-tls-certs\") pod \"test-model-9261d9-kserve-7bc497b87f-wcvlx\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") " pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.560934 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.560915    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z9zd\" (UniqueName: \"kubernetes.io/projected/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-kube-api-access-5z9zd\") pod \"test-model-9261d9-kserve-7bc497b87f-wcvlx\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") " pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.643997 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.643947    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:06.783912 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:06.779170    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"]
Aug 05 18:16:06.783912 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:16:06.783452    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4e366a3_3cc2_4600_b04b_ffb5318a4c12.slice/crio-2ae0effa284e2b390432745cac27e286f6cdff95b3f5934134ca8f2eabe59fbb WatchSource:0}: Error finding container 2ae0effa284e2b390432745cac27e286f6cdff95b3f5934134ca8f2eabe59fbb: Status 404 returned error can't find the container with id 2ae0effa284e2b390432745cac27e286f6cdff95b3f5934134ca8f2eabe59fbb
Aug 05 18:16:07.259967 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:07.259929    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx" event={"ID":"c4e366a3-3cc2-4600-b04b-ffb5318a4c12","Type":"ContainerStarted","Data":"77a706b3e444fe1f2c8a6feb51b9ac7b221cf9096918a4e59d910c51ae1e99c8"}
Aug 05 18:16:07.259967 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:07.259969    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx" event={"ID":"c4e366a3-3cc2-4600-b04b-ffb5318a4c12","Type":"ContainerStarted","Data":"2ae0effa284e2b390432745cac27e286f6cdff95b3f5934134ca8f2eabe59fbb"}
Aug 05 18:16:07.280292 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:07.280251    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx" podStartSLOduration=1.280236394 podStartE2EDuration="1.280236394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:16:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:16:07.278358295 +0000 UTC m=+2574.631032900" watchObservedRunningTime="2026-08-05 18:16:07.280236394 +0000 UTC m=+2574.632911059"
Aug 05 18:16:16.644244 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:16.644209    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:16.644741 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:16.644258    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:16.658093 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:16.658068    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:17.326788 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:17.326765    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:16:20.399512 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:20.399469    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:16:20.482067 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:20.482036    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:16:24.279761 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:24.279728    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:16:50.950300 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:16:50.950259    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:07.579148 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:07.579109    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"]
Aug 05 18:17:07.602067 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:07.602031    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:08.548018 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:08.547972    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:19.222401 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.222360    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"]
Aug 05 18:17:19.223012 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.222640    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4" podUID="20eaba87-48bd-4ee0-8b49-79aa2f0e541a" containerName="maas-api" containerID="cri-o://d2f6680f9f005b0e5697d27312537102c5fe4afbb1870fcef99267b04e000d59" gracePeriod=30
Aug 05 18:17:19.260918 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.260880    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:19.282475 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.282441    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt"]
Aug 05 18:17:19.283107 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.283033    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt" podUID="27c11d69-c064-49fb-8f70-758da8599957" containerName="payload-processing" containerID="cri-o://cb3541f3bf59ccfc7a8445116c6e5fb9590d6c042db3e4113ced8be954be82e3" gracePeriod=30
Aug 05 18:17:19.290379 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.290130    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd"]
Aug 05 18:17:19.290630 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.290597    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd" podUID="0776738a-0e68-4351-8cf1-9ba2e0ca2500" containerName="payload-pre-processing" containerID="cri-o://fb8907cd40afda25565b6a15c2a631dc605d68b83fd2215a36d27de650ec58a9" gracePeriod=30
Aug 05 18:17:19.581505 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.581482    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt"
Aug 05 18:17:19.585762 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.585742    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd"
Aug 05 18:17:19.636847 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.636817    2569 generic.go:356] "Generic (PLEG): container finished" podID="20eaba87-48bd-4ee0-8b49-79aa2f0e541a" containerID="d2f6680f9f005b0e5697d27312537102c5fe4afbb1870fcef99267b04e000d59" exitCode=0
Aug 05 18:17:19.637020 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.636892    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4" event={"ID":"20eaba87-48bd-4ee0-8b49-79aa2f0e541a","Type":"ContainerDied","Data":"d2f6680f9f005b0e5697d27312537102c5fe4afbb1870fcef99267b04e000d59"}
Aug 05 18:17:19.638342 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.638318    2569 generic.go:356] "Generic (PLEG): container finished" podID="27c11d69-c064-49fb-8f70-758da8599957" containerID="cb3541f3bf59ccfc7a8445116c6e5fb9590d6c042db3e4113ced8be954be82e3" exitCode=0
Aug 05 18:17:19.638467 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.638366    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt"
Aug 05 18:17:19.638467 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.638395    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt" event={"ID":"27c11d69-c064-49fb-8f70-758da8599957","Type":"ContainerDied","Data":"cb3541f3bf59ccfc7a8445116c6e5fb9590d6c042db3e4113ced8be954be82e3"}
Aug 05 18:17:19.638467 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.638441    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt" event={"ID":"27c11d69-c064-49fb-8f70-758da8599957","Type":"ContainerDied","Data":"3670404e13df5029201eed52064a5170c5185815ea1f30f146cd7cf4a789546e"}
Aug 05 18:17:19.638631 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.638472    2569 scope.go:117] "RemoveContainer" containerID="cb3541f3bf59ccfc7a8445116c6e5fb9590d6c042db3e4113ced8be954be82e3"
Aug 05 18:17:19.639795 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.639775    2569 generic.go:356] "Generic (PLEG): container finished" podID="0776738a-0e68-4351-8cf1-9ba2e0ca2500" containerID="fb8907cd40afda25565b6a15c2a631dc605d68b83fd2215a36d27de650ec58a9" exitCode=0
Aug 05 18:17:19.639901 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.639819    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd"
Aug 05 18:17:19.639901 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.639848    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd" event={"ID":"0776738a-0e68-4351-8cf1-9ba2e0ca2500","Type":"ContainerDied","Data":"fb8907cd40afda25565b6a15c2a631dc605d68b83fd2215a36d27de650ec58a9"}
Aug 05 18:17:19.639901 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.639888    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd" event={"ID":"0776738a-0e68-4351-8cf1-9ba2e0ca2500","Type":"ContainerDied","Data":"74203e89efcfba0bc022abf79564fef3f11e21efc738c17ad5b5267e854e223d"}
Aug 05 18:17:19.649399 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.649378    2569 scope.go:117] "RemoveContainer" containerID="cb3541f3bf59ccfc7a8445116c6e5fb9590d6c042db3e4113ced8be954be82e3"
Aug 05 18:17:19.649796 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:17:19.649773    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb3541f3bf59ccfc7a8445116c6e5fb9590d6c042db3e4113ced8be954be82e3\": container with ID starting with cb3541f3bf59ccfc7a8445116c6e5fb9590d6c042db3e4113ced8be954be82e3 not found: ID does not exist" containerID="cb3541f3bf59ccfc7a8445116c6e5fb9590d6c042db3e4113ced8be954be82e3"
Aug 05 18:17:19.649910 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.649808    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb3541f3bf59ccfc7a8445116c6e5fb9590d6c042db3e4113ced8be954be82e3"} err="failed to get container status \"cb3541f3bf59ccfc7a8445116c6e5fb9590d6c042db3e4113ced8be954be82e3\": rpc error: code = NotFound desc = could not find container \"cb3541f3bf59ccfc7a8445116c6e5fb9590d6c042db3e4113ced8be954be82e3\": container with ID starting with cb3541f3bf59ccfc7a8445116c6e5fb9590d6c042db3e4113ced8be954be82e3 not found: ID does not exist"
Aug 05 18:17:19.649910 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.649832    2569 scope.go:117] "RemoveContainer" containerID="fb8907cd40afda25565b6a15c2a631dc605d68b83fd2215a36d27de650ec58a9"
Aug 05 18:17:19.660281 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.660256    2569 scope.go:117] "RemoveContainer" containerID="fb8907cd40afda25565b6a15c2a631dc605d68b83fd2215a36d27de650ec58a9"
Aug 05 18:17:19.660811 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:17:19.660790    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb8907cd40afda25565b6a15c2a631dc605d68b83fd2215a36d27de650ec58a9\": container with ID starting with fb8907cd40afda25565b6a15c2a631dc605d68b83fd2215a36d27de650ec58a9 not found: ID does not exist" containerID="fb8907cd40afda25565b6a15c2a631dc605d68b83fd2215a36d27de650ec58a9"
Aug 05 18:17:19.660905 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.660817    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb8907cd40afda25565b6a15c2a631dc605d68b83fd2215a36d27de650ec58a9"} err="failed to get container status \"fb8907cd40afda25565b6a15c2a631dc605d68b83fd2215a36d27de650ec58a9\": rpc error: code = NotFound desc = could not find container \"fb8907cd40afda25565b6a15c2a631dc605d68b83fd2215a36d27de650ec58a9\": container with ID starting with fb8907cd40afda25565b6a15c2a631dc605d68b83fd2215a36d27de650ec58a9 not found: ID does not exist"
Aug 05 18:17:19.676091 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.676069    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"
Aug 05 18:17:19.715330 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.715298    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27c11d69-c064-49fb-8f70-758da8599957-plugins-config-volume\") pod \"27c11d69-c064-49fb-8f70-758da8599957\" (UID: \"27c11d69-c064-49fb-8f70-758da8599957\") "
Aug 05 18:17:19.715330 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.715334    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20eaba87-48bd-4ee0-8b49-79aa2f0e541a-openshift-service-ca\") pod \"20eaba87-48bd-4ee0-8b49-79aa2f0e541a\" (UID: \"20eaba87-48bd-4ee0-8b49-79aa2f0e541a\") "
Aug 05 18:17:19.715550 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.715374    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0776738a-0e68-4351-8cf1-9ba2e0ca2500-plugins-config-volume\") pod \"0776738a-0e68-4351-8cf1-9ba2e0ca2500\" (UID: \"0776738a-0e68-4351-8cf1-9ba2e0ca2500\") "
Aug 05 18:17:19.715550 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.715409    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x855n\" (UniqueName: \"kubernetes.io/projected/0776738a-0e68-4351-8cf1-9ba2e0ca2500-kube-api-access-x855n\") pod \"0776738a-0e68-4351-8cf1-9ba2e0ca2500\" (UID: \"0776738a-0e68-4351-8cf1-9ba2e0ca2500\") "
Aug 05 18:17:19.715550 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.715439    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjzkq\" (UniqueName: \"kubernetes.io/projected/27c11d69-c064-49fb-8f70-758da8599957-kube-api-access-hjzkq\") pod \"27c11d69-c064-49fb-8f70-758da8599957\" (UID: \"27c11d69-c064-49fb-8f70-758da8599957\") "
Aug 05 18:17:19.715550 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.715465    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mhbt\" (UniqueName: \"kubernetes.io/projected/20eaba87-48bd-4ee0-8b49-79aa2f0e541a-kube-api-access-7mhbt\") pod \"20eaba87-48bd-4ee0-8b49-79aa2f0e541a\" (UID: \"20eaba87-48bd-4ee0-8b49-79aa2f0e541a\") "
Aug 05 18:17:19.715550 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.715525    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20eaba87-48bd-4ee0-8b49-79aa2f0e541a-maas-api-tls\") pod \"20eaba87-48bd-4ee0-8b49-79aa2f0e541a\" (UID: \"20eaba87-48bd-4ee0-8b49-79aa2f0e541a\") "
Aug 05 18:17:19.715809 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.715712    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27c11d69-c064-49fb-8f70-758da8599957-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "27c11d69-c064-49fb-8f70-758da8599957" (UID: "27c11d69-c064-49fb-8f70-758da8599957"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:17:19.715809 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.715734    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20eaba87-48bd-4ee0-8b49-79aa2f0e541a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "20eaba87-48bd-4ee0-8b49-79aa2f0e541a" (UID: "20eaba87-48bd-4ee0-8b49-79aa2f0e541a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:17:19.715914 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.715855    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0776738a-0e68-4351-8cf1-9ba2e0ca2500-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0776738a-0e68-4351-8cf1-9ba2e0ca2500" (UID: "0776738a-0e68-4351-8cf1-9ba2e0ca2500"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:17:19.715914 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.715864    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27c11d69-c064-49fb-8f70-758da8599957-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:17:19.715914 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.715890    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20eaba87-48bd-4ee0-8b49-79aa2f0e541a-openshift-service-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:17:19.718118 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.718088    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27c11d69-c064-49fb-8f70-758da8599957-kube-api-access-hjzkq" (OuterVolumeSpecName: "kube-api-access-hjzkq") pod "27c11d69-c064-49fb-8f70-758da8599957" (UID: "27c11d69-c064-49fb-8f70-758da8599957"). InnerVolumeSpecName "kube-api-access-hjzkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:17:19.718294 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.718158    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0776738a-0e68-4351-8cf1-9ba2e0ca2500-kube-api-access-x855n" (OuterVolumeSpecName: "kube-api-access-x855n") pod "0776738a-0e68-4351-8cf1-9ba2e0ca2500" (UID: "0776738a-0e68-4351-8cf1-9ba2e0ca2500"). InnerVolumeSpecName "kube-api-access-x855n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:17:19.718294 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.718208    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20eaba87-48bd-4ee0-8b49-79aa2f0e541a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "20eaba87-48bd-4ee0-8b49-79aa2f0e541a" (UID: "20eaba87-48bd-4ee0-8b49-79aa2f0e541a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:17:19.718424 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.718406    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20eaba87-48bd-4ee0-8b49-79aa2f0e541a-kube-api-access-7mhbt" (OuterVolumeSpecName: "kube-api-access-7mhbt") pod "20eaba87-48bd-4ee0-8b49-79aa2f0e541a" (UID: "20eaba87-48bd-4ee0-8b49-79aa2f0e541a"). InnerVolumeSpecName "kube-api-access-7mhbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:17:19.817379 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.817352    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20eaba87-48bd-4ee0-8b49-79aa2f0e541a-maas-api-tls\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:17:19.817379 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.817378    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0776738a-0e68-4351-8cf1-9ba2e0ca2500-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:17:19.817572 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.817388    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x855n\" (UniqueName: \"kubernetes.io/projected/0776738a-0e68-4351-8cf1-9ba2e0ca2500-kube-api-access-x855n\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:17:19.817572 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.817398    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjzkq\" (UniqueName: \"kubernetes.io/projected/27c11d69-c064-49fb-8f70-758da8599957-kube-api-access-hjzkq\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:17:19.817572 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.817407    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mhbt\" (UniqueName: \"kubernetes.io/projected/20eaba87-48bd-4ee0-8b49-79aa2f0e541a-kube-api-access-7mhbt\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:17:19.979017 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.978981    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt"]
Aug 05 18:17:19.992006 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:19.991976    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9261d9-64994d49b8-dpntt"]
Aug 05 18:17:20.016133 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:20.016098    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd"]
Aug 05 18:17:20.020773 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:20.020748    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9261d9-7c7d85b47b-f4qcd"]
Aug 05 18:17:20.647952 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:20.647925    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"
Aug 05 18:17:20.647952 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:20.647941    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4" event={"ID":"20eaba87-48bd-4ee0-8b49-79aa2f0e541a","Type":"ContainerDied","Data":"84c52b53be053c5800ded5bc24f0d3eb836004e4e216a2760b503ce820d46f69"}
Aug 05 18:17:20.648514 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:20.648016    2569 scope.go:117] "RemoveContainer" containerID="d2f6680f9f005b0e5697d27312537102c5fe4afbb1870fcef99267b04e000d59"
Aug 05 18:17:20.670720 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:20.670698    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"]
Aug 05 18:17:20.676667 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:20.676645    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9261d9-68f6bc8575-fhwt4"]
Aug 05 18:17:20.731020 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:20.730999    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:21.152981 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:21.152940    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0776738a-0e68-4351-8cf1-9ba2e0ca2500" path="/var/lib/kubelet/pods/0776738a-0e68-4351-8cf1-9ba2e0ca2500/volumes"
Aug 05 18:17:21.153466 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:21.153445    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20eaba87-48bd-4ee0-8b49-79aa2f0e541a" path="/var/lib/kubelet/pods/20eaba87-48bd-4ee0-8b49-79aa2f0e541a/volumes"
Aug 05 18:17:21.153818 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:21.153800    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27c11d69-c064-49fb-8f70-758da8599957" path="/var/lib/kubelet/pods/27c11d69-c064-49fb-8f70-758da8599957/volumes"
Aug 05 18:17:22.600726 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:22.600666    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx" podUID="c4e366a3-3cc2-4600-b04b-ffb5318a4c12" containerName="main" containerID="cri-o://77a706b3e444fe1f2c8a6feb51b9ac7b221cf9096918a4e59d910c51ae1e99c8" gracePeriod=45
Aug 05 18:17:22.858043 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:22.857977    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:17:22.945375 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:22.945347    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-dshm\") pod \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") "
Aug 05 18:17:22.945533 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:22.945414    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-tls-certs\") pod \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") "
Aug 05 18:17:22.945533 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:22.945432    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-tmp-dir\") pod \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") "
Aug 05 18:17:22.945533 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:22.945456    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5z9zd\" (UniqueName: \"kubernetes.io/projected/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-kube-api-access-5z9zd\") pod \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") "
Aug 05 18:17:22.945533 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:22.945482    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-model-cache\") pod \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") "
Aug 05 18:17:22.945533 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:22.945511    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-home\") pod \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\" (UID: \"c4e366a3-3cc2-4600-b04b-ffb5318a4c12\") "
Aug 05 18:17:22.945932 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:22.945848    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c4e366a3-3cc2-4600-b04b-ffb5318a4c12" (UID: "c4e366a3-3cc2-4600-b04b-ffb5318a4c12"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:22.945932 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:22.945883    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-model-cache" (OuterVolumeSpecName: "model-cache") pod "c4e366a3-3cc2-4600-b04b-ffb5318a4c12" (UID: "c4e366a3-3cc2-4600-b04b-ffb5318a4c12"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:22.946084 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:22.945949    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-home" (OuterVolumeSpecName: "home") pod "c4e366a3-3cc2-4600-b04b-ffb5318a4c12" (UID: "c4e366a3-3cc2-4600-b04b-ffb5318a4c12"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:22.947827 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:22.947799    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-dshm" (OuterVolumeSpecName: "dshm") pod "c4e366a3-3cc2-4600-b04b-ffb5318a4c12" (UID: "c4e366a3-3cc2-4600-b04b-ffb5318a4c12"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:22.947827 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:22.947810    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-kube-api-access-5z9zd" (OuterVolumeSpecName: "kube-api-access-5z9zd") pod "c4e366a3-3cc2-4600-b04b-ffb5318a4c12" (UID: "c4e366a3-3cc2-4600-b04b-ffb5318a4c12"). InnerVolumeSpecName "kube-api-access-5z9zd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:17:22.947953 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:22.947857    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c4e366a3-3cc2-4600-b04b-ffb5318a4c12" (UID: "c4e366a3-3cc2-4600-b04b-ffb5318a4c12"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:17:23.046273 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:23.046239    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-dshm\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:17:23.046273 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:23.046270    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-tls-certs\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:17:23.046273 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:23.046282    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-tmp-dir\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:17:23.046482 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:23.046290    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5z9zd\" (UniqueName: \"kubernetes.io/projected/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-kube-api-access-5z9zd\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:17:23.046482 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:23.046299    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-model-cache\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:17:23.046482 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:23.046307    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4e366a3-3cc2-4600-b04b-ffb5318a4c12-home\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:17:23.665738 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:23.665705    2569 generic.go:356] "Generic (PLEG): container finished" podID="c4e366a3-3cc2-4600-b04b-ffb5318a4c12" containerID="77a706b3e444fe1f2c8a6feb51b9ac7b221cf9096918a4e59d910c51ae1e99c8" exitCode=0
Aug 05 18:17:23.666193 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:23.665751    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"
Aug 05 18:17:23.666193 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:23.665790    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx" event={"ID":"c4e366a3-3cc2-4600-b04b-ffb5318a4c12","Type":"ContainerDied","Data":"77a706b3e444fe1f2c8a6feb51b9ac7b221cf9096918a4e59d910c51ae1e99c8"}
Aug 05 18:17:23.666193 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:23.665831    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx" event={"ID":"c4e366a3-3cc2-4600-b04b-ffb5318a4c12","Type":"ContainerDied","Data":"2ae0effa284e2b390432745cac27e286f6cdff95b3f5934134ca8f2eabe59fbb"}
Aug 05 18:17:23.666193 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:23.665848    2569 scope.go:117] "RemoveContainer" containerID="77a706b3e444fe1f2c8a6feb51b9ac7b221cf9096918a4e59d910c51ae1e99c8"
Aug 05 18:17:23.675000 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:23.674971    2569 scope.go:117] "RemoveContainer" containerID="77a706b3e444fe1f2c8a6feb51b9ac7b221cf9096918a4e59d910c51ae1e99c8"
Aug 05 18:17:23.675351 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:17:23.675332    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77a706b3e444fe1f2c8a6feb51b9ac7b221cf9096918a4e59d910c51ae1e99c8\": container with ID starting with 77a706b3e444fe1f2c8a6feb51b9ac7b221cf9096918a4e59d910c51ae1e99c8 not found: ID does not exist" containerID="77a706b3e444fe1f2c8a6feb51b9ac7b221cf9096918a4e59d910c51ae1e99c8"
Aug 05 18:17:23.675429 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:23.675357    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77a706b3e444fe1f2c8a6feb51b9ac7b221cf9096918a4e59d910c51ae1e99c8"} err="failed to get container status \"77a706b3e444fe1f2c8a6feb51b9ac7b221cf9096918a4e59d910c51ae1e99c8\": rpc error: code = NotFound desc = could not find container \"77a706b3e444fe1f2c8a6feb51b9ac7b221cf9096918a4e59d910c51ae1e99c8\": container with ID starting with 77a706b3e444fe1f2c8a6feb51b9ac7b221cf9096918a4e59d910c51ae1e99c8 not found: ID does not exist"
Aug 05 18:17:23.683134 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:23.683112    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"]
Aug 05 18:17:23.687112 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:23.687090    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-9261d9/test-model-9261d9-kserve-7bc497b87f-wcvlx"]
Aug 05 18:17:25.151536 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:25.151503    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4e366a3-3cc2-4600-b04b-ffb5318a4c12" path="/var/lib/kubelet/pods/c4e366a3-3cc2-4600-b04b-ffb5318a4c12/volumes"
Aug 05 18:17:26.039981 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:26.039940    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:26.137042 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:26.137010    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:26.299991 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:26.299894    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:26.500226 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:26.500164    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:37.212952 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.212920    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:37.235188 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.235144    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv"]
Aug 05 18:17:37.235443 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.235395    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv" podUID="c111cc1d-00dc-4d8c-8fc4-09167a862dd3" containerName="payload-pre-processing" containerID="cri-o://0c3f485b7f72c19cbf3e75ec2e00ff226500d1e08e3bc42796031b09f08d7f01" gracePeriod=30
Aug 05 18:17:37.306624 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.306592    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:37.403411 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.403384    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:37.500305 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.500280    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv"
Aug 05 18:17:37.571726 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.571689    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blv5r\" (UniqueName: \"kubernetes.io/projected/c111cc1d-00dc-4d8c-8fc4-09167a862dd3-kube-api-access-blv5r\") pod \"c111cc1d-00dc-4d8c-8fc4-09167a862dd3\" (UID: \"c111cc1d-00dc-4d8c-8fc4-09167a862dd3\") "
Aug 05 18:17:37.571875 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.571780    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c111cc1d-00dc-4d8c-8fc4-09167a862dd3-plugins-config-volume\") pod \"c111cc1d-00dc-4d8c-8fc4-09167a862dd3\" (UID: \"c111cc1d-00dc-4d8c-8fc4-09167a862dd3\") "
Aug 05 18:17:37.572143 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.572113    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c111cc1d-00dc-4d8c-8fc4-09167a862dd3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c111cc1d-00dc-4d8c-8fc4-09167a862dd3" (UID: "c111cc1d-00dc-4d8c-8fc4-09167a862dd3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:17:37.573741 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.573718    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c111cc1d-00dc-4d8c-8fc4-09167a862dd3-kube-api-access-blv5r" (OuterVolumeSpecName: "kube-api-access-blv5r") pod "c111cc1d-00dc-4d8c-8fc4-09167a862dd3" (UID: "c111cc1d-00dc-4d8c-8fc4-09167a862dd3"). InnerVolumeSpecName "kube-api-access-blv5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:17:37.672785 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.672761    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blv5r\" (UniqueName: \"kubernetes.io/projected/c111cc1d-00dc-4d8c-8fc4-09167a862dd3-kube-api-access-blv5r\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:17:37.672895 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.672786    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c111cc1d-00dc-4d8c-8fc4-09167a862dd3-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:17:37.741437 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.741369    2569 generic.go:356] "Generic (PLEG): container finished" podID="c111cc1d-00dc-4d8c-8fc4-09167a862dd3" containerID="0c3f485b7f72c19cbf3e75ec2e00ff226500d1e08e3bc42796031b09f08d7f01" exitCode=0
Aug 05 18:17:37.741437 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.741426    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv"
Aug 05 18:17:37.741607 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.741458    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv" event={"ID":"c111cc1d-00dc-4d8c-8fc4-09167a862dd3","Type":"ContainerDied","Data":"0c3f485b7f72c19cbf3e75ec2e00ff226500d1e08e3bc42796031b09f08d7f01"}
Aug 05 18:17:37.741607 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.741533    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv" event={"ID":"c111cc1d-00dc-4d8c-8fc4-09167a862dd3","Type":"ContainerDied","Data":"a46175e01c9b6c7435f58a1bedd1491784f043ab404f0737db65e5fae185cecf"}
Aug 05 18:17:37.741607 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.741556    2569 scope.go:117] "RemoveContainer" containerID="0c3f485b7f72c19cbf3e75ec2e00ff226500d1e08e3bc42796031b09f08d7f01"
Aug 05 18:17:37.751929 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.751906    2569 scope.go:117] "RemoveContainer" containerID="0c3f485b7f72c19cbf3e75ec2e00ff226500d1e08e3bc42796031b09f08d7f01"
Aug 05 18:17:37.752205 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:17:37.752162    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c3f485b7f72c19cbf3e75ec2e00ff226500d1e08e3bc42796031b09f08d7f01\": container with ID starting with 0c3f485b7f72c19cbf3e75ec2e00ff226500d1e08e3bc42796031b09f08d7f01 not found: ID does not exist" containerID="0c3f485b7f72c19cbf3e75ec2e00ff226500d1e08e3bc42796031b09f08d7f01"
Aug 05 18:17:37.752282 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.752216    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c3f485b7f72c19cbf3e75ec2e00ff226500d1e08e3bc42796031b09f08d7f01"} err="failed to get container status \"0c3f485b7f72c19cbf3e75ec2e00ff226500d1e08e3bc42796031b09f08d7f01\": rpc error: code = NotFound desc = could not find container \"0c3f485b7f72c19cbf3e75ec2e00ff226500d1e08e3bc42796031b09f08d7f01\": container with ID starting with 0c3f485b7f72c19cbf3e75ec2e00ff226500d1e08e3bc42796031b09f08d7f01 not found: ID does not exist"
Aug 05 18:17:37.766790 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.766771    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv"]
Aug 05 18:17:37.770548 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.770527    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:37.776504 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:37.776477    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-955c47-75bd4cc456-w9bpv"]
Aug 05 18:17:39.151677 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:39.151647    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c111cc1d-00dc-4d8c-8fc4-09167a862dd3" path="/var/lib/kubelet/pods/c111cc1d-00dc-4d8c-8fc4-09167a862dd3/volumes"
Aug 05 18:17:43.843507 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:43.843468    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:43.943483 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:43.943448    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:44.131559 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:44.131439    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:44.327669 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:44.327637    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:44.659972 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:44.659940    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:44.949341 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:44.949224    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:45.330598 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:45.330568    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:47.922101 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:47.922064    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:48.043359 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:48.043323    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:49.060636 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.060603    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"]
Aug 05 18:17:49.061087 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.061014    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27c11d69-c064-49fb-8f70-758da8599957" containerName="payload-processing"
Aug 05 18:17:49.061087 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.061027    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="27c11d69-c064-49fb-8f70-758da8599957" containerName="payload-processing"
Aug 05 18:17:49.061087 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.061059    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20eaba87-48bd-4ee0-8b49-79aa2f0e541a" containerName="maas-api"
Aug 05 18:17:49.061087 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.061068    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="20eaba87-48bd-4ee0-8b49-79aa2f0e541a" containerName="maas-api"
Aug 05 18:17:49.061087 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.061078    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0776738a-0e68-4351-8cf1-9ba2e0ca2500" containerName="payload-pre-processing"
Aug 05 18:17:49.061087 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.061086    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0776738a-0e68-4351-8cf1-9ba2e0ca2500" containerName="payload-pre-processing"
Aug 05 18:17:49.061377 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.061103    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c111cc1d-00dc-4d8c-8fc4-09167a862dd3" containerName="payload-pre-processing"
Aug 05 18:17:49.061377 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.061111    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c111cc1d-00dc-4d8c-8fc4-09167a862dd3" containerName="payload-pre-processing"
Aug 05 18:17:49.061377 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.061119    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4e366a3-3cc2-4600-b04b-ffb5318a4c12" containerName="main"
Aug 05 18:17:49.061377 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.061127    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4e366a3-3cc2-4600-b04b-ffb5318a4c12" containerName="main"
Aug 05 18:17:49.061377 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.061275    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0776738a-0e68-4351-8cf1-9ba2e0ca2500" containerName="payload-pre-processing"
Aug 05 18:17:49.061377 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.061288    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="27c11d69-c064-49fb-8f70-758da8599957" containerName="payload-processing"
Aug 05 18:17:49.061377 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.061295    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4e366a3-3cc2-4600-b04b-ffb5318a4c12" containerName="main"
Aug 05 18:17:49.061377 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.061304    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="20eaba87-48bd-4ee0-8b49-79aa2f0e541a" containerName="maas-api"
Aug 05 18:17:49.061377 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.061309    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c111cc1d-00dc-4d8c-8fc4-09167a862dd3" containerName="payload-pre-processing"
Aug 05 18:17:49.065848 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.065829    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"
Aug 05 18:17:49.068405 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.068384    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-acb45a-serving-cert\""
Aug 05 18:17:49.068520 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.068404    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jdhhr\""
Aug 05 18:17:49.077940 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.077917    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"]
Aug 05 18:17:49.090789 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.090769    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:49.170856 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.170823    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5599c65-a852-434c-ba64-cfad5d82acd2-openshift-service-ca\") pod \"maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895\" (UID: \"e5599c65-a852-434c-ba64-cfad5d82acd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"
Aug 05 18:17:49.171020 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.170884    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8spn\" (UniqueName: \"kubernetes.io/projected/e5599c65-a852-434c-ba64-cfad5d82acd2-kube-api-access-l8spn\") pod \"maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895\" (UID: \"e5599c65-a852-434c-ba64-cfad5d82acd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"
Aug 05 18:17:49.171020 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.170974    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5599c65-a852-434c-ba64-cfad5d82acd2-maas-api-tls\") pod \"maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895\" (UID: \"e5599c65-a852-434c-ba64-cfad5d82acd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"
Aug 05 18:17:49.271687 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.271657    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5599c65-a852-434c-ba64-cfad5d82acd2-openshift-service-ca\") pod \"maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895\" (UID: \"e5599c65-a852-434c-ba64-cfad5d82acd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"
Aug 05 18:17:49.271815 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.271706    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8spn\" (UniqueName: \"kubernetes.io/projected/e5599c65-a852-434c-ba64-cfad5d82acd2-kube-api-access-l8spn\") pod \"maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895\" (UID: \"e5599c65-a852-434c-ba64-cfad5d82acd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"
Aug 05 18:17:49.271815 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.271752    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5599c65-a852-434c-ba64-cfad5d82acd2-maas-api-tls\") pod \"maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895\" (UID: \"e5599c65-a852-434c-ba64-cfad5d82acd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"
Aug 05 18:17:49.271900 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:17:49.271884    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-serving-cert: secret "maas-api-e2e-api-a-acb45a-serving-cert" not found
Aug 05 18:17:49.271969 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:17:49.271959    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5599c65-a852-434c-ba64-cfad5d82acd2-maas-api-tls podName:e5599c65-a852-434c-ba64-cfad5d82acd2 nodeName:}" failed. No retries permitted until 2026-08-05 18:17:49.771937336 +0000 UTC m=+2677.124611908 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e5599c65-a852-434c-ba64-cfad5d82acd2-maas-api-tls") pod "maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895" (UID: "e5599c65-a852-434c-ba64-cfad5d82acd2") : secret "maas-api-e2e-api-a-acb45a-serving-cert" not found
Aug 05 18:17:49.272286 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.272268    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5599c65-a852-434c-ba64-cfad5d82acd2-openshift-service-ca\") pod \"maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895\" (UID: \"e5599c65-a852-434c-ba64-cfad5d82acd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"
Aug 05 18:17:49.280733 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.280710    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8spn\" (UniqueName: \"kubernetes.io/projected/e5599c65-a852-434c-ba64-cfad5d82acd2-kube-api-access-l8spn\") pod \"maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895\" (UID: \"e5599c65-a852-434c-ba64-cfad5d82acd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"
Aug 05 18:17:49.605743 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.605701    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:49.700544 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.700506    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6"]
Aug 05 18:17:49.704222 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.704203    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6"
Aug 05 18:17:49.706626 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.706606    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-acb45a\""
Aug 05 18:17:49.707224 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.707205    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-acb45a-dockercfg-kfc4l\""
Aug 05 18:17:49.718326 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.718264    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:49.718796 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.718776    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6"]
Aug 05 18:17:49.734627 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.734605    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb"]
Aug 05 18:17:49.738306 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.738290    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb"
Aug 05 18:17:49.752232 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.752209    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb"]
Aug 05 18:17:49.777885 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.777860    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5599c65-a852-434c-ba64-cfad5d82acd2-maas-api-tls\") pod \"maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895\" (UID: \"e5599c65-a852-434c-ba64-cfad5d82acd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"
Aug 05 18:17:49.780668 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.780647    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5599c65-a852-434c-ba64-cfad5d82acd2-maas-api-tls\") pod \"maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895\" (UID: \"e5599c65-a852-434c-ba64-cfad5d82acd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"
Aug 05 18:17:49.879447 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.879362    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgjdj\" (UniqueName: \"kubernetes.io/projected/5f75f09b-e111-4be0-b410-f45283a55b41-kube-api-access-lgjdj\") pod \"payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb\" (UID: \"5f75f09b-e111-4be0-b410-f45283a55b41\") " pod="openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb"
Aug 05 18:17:49.879447 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.879411    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f75f09b-e111-4be0-b410-f45283a55b41-plugins-config-volume\") pod \"payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb\" (UID: \"5f75f09b-e111-4be0-b410-f45283a55b41\") " pod="openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb"
Aug 05 18:17:49.879655 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.879486    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtzkb\" (UniqueName: \"kubernetes.io/projected/65972304-ea5f-40fe-a7f1-4e9e283a8f93-kube-api-access-dtzkb\") pod \"payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6\" (UID: \"65972304-ea5f-40fe-a7f1-4e9e283a8f93\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6"
Aug 05 18:17:49.879655 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.879534    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65972304-ea5f-40fe-a7f1-4e9e283a8f93-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6\" (UID: \"65972304-ea5f-40fe-a7f1-4e9e283a8f93\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6"
Aug 05 18:17:49.977524 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.977487    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"
Aug 05 18:17:49.980524 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.980166    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgjdj\" (UniqueName: \"kubernetes.io/projected/5f75f09b-e111-4be0-b410-f45283a55b41-kube-api-access-lgjdj\") pod \"payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb\" (UID: \"5f75f09b-e111-4be0-b410-f45283a55b41\") " pod="openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb"
Aug 05 18:17:49.980524 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.980250    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f75f09b-e111-4be0-b410-f45283a55b41-plugins-config-volume\") pod \"payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb\" (UID: \"5f75f09b-e111-4be0-b410-f45283a55b41\") " pod="openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb"
Aug 05 18:17:49.980524 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.980291    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtzkb\" (UniqueName: \"kubernetes.io/projected/65972304-ea5f-40fe-a7f1-4e9e283a8f93-kube-api-access-dtzkb\") pod \"payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6\" (UID: \"65972304-ea5f-40fe-a7f1-4e9e283a8f93\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6"
Aug 05 18:17:49.980524 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.980340    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65972304-ea5f-40fe-a7f1-4e9e283a8f93-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6\" (UID: \"65972304-ea5f-40fe-a7f1-4e9e283a8f93\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6"
Aug 05 18:17:49.981017 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.980987    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65972304-ea5f-40fe-a7f1-4e9e283a8f93-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6\" (UID: \"65972304-ea5f-40fe-a7f1-4e9e283a8f93\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6"
Aug 05 18:17:49.981092 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.981027    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f75f09b-e111-4be0-b410-f45283a55b41-plugins-config-volume\") pod \"payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb\" (UID: \"5f75f09b-e111-4be0-b410-f45283a55b41\") " pod="openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb"
Aug 05 18:17:49.989381 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.989354    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgjdj\" (UniqueName: \"kubernetes.io/projected/5f75f09b-e111-4be0-b410-f45283a55b41-kube-api-access-lgjdj\") pod \"payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb\" (UID: \"5f75f09b-e111-4be0-b410-f45283a55b41\") " pod="openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb"
Aug 05 18:17:49.989468 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:49.989415    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtzkb\" (UniqueName: \"kubernetes.io/projected/65972304-ea5f-40fe-a7f1-4e9e283a8f93-kube-api-access-dtzkb\") pod \"payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6\" (UID: \"65972304-ea5f-40fe-a7f1-4e9e283a8f93\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6"
Aug 05 18:17:50.015557 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:50.015516    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6"
Aug 05 18:17:50.048611 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:50.048355    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb"
Aug 05 18:17:50.148478 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:50.148359    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"]
Aug 05 18:17:50.153329 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:17:50.153272    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5599c65_a852_434c_ba64_cfad5d82acd2.slice/crio-3d98963e4665299d14d2324799f787f135dca04011f0079a40fab12d7e02e885 WatchSource:0}: Error finding container 3d98963e4665299d14d2324799f787f135dca04011f0079a40fab12d7e02e885: Status 404 returned error can't find the container with id 3d98963e4665299d14d2324799f787f135dca04011f0079a40fab12d7e02e885
Aug 05 18:17:50.155064 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:50.155041    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 18:17:50.194773 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:50.194745    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6"]
Aug 05 18:17:50.196882 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:17:50.196844    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65972304_ea5f_40fe_a7f1_4e9e283a8f93.slice/crio-556ac62112985d27e3afad0da70f777a3741d24132fe8875298853d14a2205bc WatchSource:0}: Error finding container 556ac62112985d27e3afad0da70f777a3741d24132fe8875298853d14a2205bc: Status 404 returned error can't find the container with id 556ac62112985d27e3afad0da70f777a3741d24132fe8875298853d14a2205bc
Aug 05 18:17:50.224435 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:50.224413    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb"]
Aug 05 18:17:50.226402 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:17:50.226379    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f75f09b_e111_4be0_b410_f45283a55b41.slice/crio-73407f0d8927285cef4d6772f7129342250da36634ab7eb8640907346c0c0556 WatchSource:0}: Error finding container 73407f0d8927285cef4d6772f7129342250da36634ab7eb8640907346c0c0556: Status 404 returned error can't find the container with id 73407f0d8927285cef4d6772f7129342250da36634ab7eb8640907346c0c0556
Aug 05 18:17:50.811917 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:50.811883    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb" event={"ID":"5f75f09b-e111-4be0-b410-f45283a55b41","Type":"ContainerStarted","Data":"e499e8629c688df9636808fca144abf2945e68e287d1495a8ad819cbc67378d1"}
Aug 05 18:17:50.812044 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:50.811943    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb" event={"ID":"5f75f09b-e111-4be0-b410-f45283a55b41","Type":"ContainerStarted","Data":"73407f0d8927285cef4d6772f7129342250da36634ab7eb8640907346c0c0556"}
Aug 05 18:17:50.812044 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:50.811972    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb"
Aug 05 18:17:50.813516 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:50.813487    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6" event={"ID":"65972304-ea5f-40fe-a7f1-4e9e283a8f93","Type":"ContainerStarted","Data":"db85528eb4fc90a626bd71aea78036c42f5383c3241c7d6fe7b0ad29cce5ca41"}
Aug 05 18:17:50.813614 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:50.813542    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6" event={"ID":"65972304-ea5f-40fe-a7f1-4e9e283a8f93","Type":"ContainerStarted","Data":"556ac62112985d27e3afad0da70f777a3741d24132fe8875298853d14a2205bc"}
Aug 05 18:17:50.813614 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:50.813566    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6"
Aug 05 18:17:50.815033 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:50.815007    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895" event={"ID":"e5599c65-a852-434c-ba64-cfad5d82acd2","Type":"ContainerStarted","Data":"2d134d1eab6521caeedc103612a0ff7bf01fe62f44ff283e41197fd862f6b0b6"}
Aug 05 18:17:50.815133 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:50.815046    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895" event={"ID":"e5599c65-a852-434c-ba64-cfad5d82acd2","Type":"ContainerStarted","Data":"3d98963e4665299d14d2324799f787f135dca04011f0079a40fab12d7e02e885"}
Aug 05 18:17:50.815133 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:50.815066    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"
Aug 05 18:17:50.848329 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:50.848289    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb" podStartSLOduration=1.848274797 podStartE2EDuration="1.848274797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:17:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:17:50.847851026 +0000 UTC m=+2678.200525622" watchObservedRunningTime="2026-08-05 18:17:50.848274797 +0000 UTC m=+2678.200949420"
Aug 05 18:17:50.881044 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:50.881008    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6" podStartSLOduration=1.880987684 podStartE2EDuration="1.880987684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:17:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:17:50.880069481 +0000 UTC m=+2678.232744075" watchObservedRunningTime="2026-08-05 18:17:50.880987684 +0000 UTC m=+2678.233662255"
Aug 05 18:17:50.904082 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:50.904032    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895" podStartSLOduration=1.36864006 podStartE2EDuration="1.904012875s" totalImagesPullingTime="535.372815ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:17:50.901810992 +0000 UTC m=+2678.254485586" watchObservedRunningTime="2026-08-05 18:17:50.904012875 +0000 UTC m=+2678.256687471"
Aug 05 18:17:56.825389 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:17:56.825363    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"
Aug 05 18:18:01.822285 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:01.822253    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6"
Aug 05 18:18:01.822735 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:01.822320    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb"
Aug 05 18:18:12.841363 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:12.841290    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:18:13.345935 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:13.345881    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:18:13.721238 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:13.721124    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:18:14.140164 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:14.140132    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:18:17.041394 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:17.041355    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:18:17.180893 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:17.180862    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:18:18.108560 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.108520    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"]
Aug 05 18:18:18.113698 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.113676    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"
Aug 05 18:18:18.116297 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.116281    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-87a594-serving-cert\""
Aug 05 18:18:18.124731 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.124705    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"]
Aug 05 18:18:18.148985 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.148962    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:18:18.219380 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.219359    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg2vg\" (UniqueName: \"kubernetes.io/projected/180d876b-8d08-4d92-afe3-de4359cadd42-kube-api-access-jg2vg\") pod \"maas-api-e2e-api-b-87a594-84b48744d6-2psrb\" (UID: \"180d876b-8d08-4d92-afe3-de4359cadd42\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"
Aug 05 18:18:18.219496 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.219421    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/180d876b-8d08-4d92-afe3-de4359cadd42-openshift-service-ca\") pod \"maas-api-e2e-api-b-87a594-84b48744d6-2psrb\" (UID: \"180d876b-8d08-4d92-afe3-de4359cadd42\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"
Aug 05 18:18:18.219568 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.219545    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/180d876b-8d08-4d92-afe3-de4359cadd42-maas-api-tls\") pod \"maas-api-e2e-api-b-87a594-84b48744d6-2psrb\" (UID: \"180d876b-8d08-4d92-afe3-de4359cadd42\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"
Aug 05 18:18:18.320310 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.320284    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg2vg\" (UniqueName: \"kubernetes.io/projected/180d876b-8d08-4d92-afe3-de4359cadd42-kube-api-access-jg2vg\") pod \"maas-api-e2e-api-b-87a594-84b48744d6-2psrb\" (UID: \"180d876b-8d08-4d92-afe3-de4359cadd42\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"
Aug 05 18:18:18.320417 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.320344    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/180d876b-8d08-4d92-afe3-de4359cadd42-openshift-service-ca\") pod \"maas-api-e2e-api-b-87a594-84b48744d6-2psrb\" (UID: \"180d876b-8d08-4d92-afe3-de4359cadd42\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"
Aug 05 18:18:18.320417 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.320391    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/180d876b-8d08-4d92-afe3-de4359cadd42-maas-api-tls\") pod \"maas-api-e2e-api-b-87a594-84b48744d6-2psrb\" (UID: \"180d876b-8d08-4d92-afe3-de4359cadd42\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"
Aug 05 18:18:18.320989 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.320968    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/180d876b-8d08-4d92-afe3-de4359cadd42-openshift-service-ca\") pod \"maas-api-e2e-api-b-87a594-84b48744d6-2psrb\" (UID: \"180d876b-8d08-4d92-afe3-de4359cadd42\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"
Aug 05 18:18:18.323015 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.322988    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/180d876b-8d08-4d92-afe3-de4359cadd42-maas-api-tls\") pod \"maas-api-e2e-api-b-87a594-84b48744d6-2psrb\" (UID: \"180d876b-8d08-4d92-afe3-de4359cadd42\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"
Aug 05 18:18:18.329742 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.329719    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg2vg\" (UniqueName: \"kubernetes.io/projected/180d876b-8d08-4d92-afe3-de4359cadd42-kube-api-access-jg2vg\") pod \"maas-api-e2e-api-b-87a594-84b48744d6-2psrb\" (UID: \"180d876b-8d08-4d92-afe3-de4359cadd42\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"
Aug 05 18:18:18.370870 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.370802    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:18:18.425292 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.425269    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"
Aug 05 18:18:18.569570 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.569539    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"]
Aug 05 18:18:18.570991 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:18:18.570967    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod180d876b_8d08_4d92_afe3_de4359cadd42.slice/crio-8985d94af986db7446a6967ff7b39f6220cefe152e4e4a63ad37b1c86bbf1409 WatchSource:0}: Error finding container 8985d94af986db7446a6967ff7b39f6220cefe152e4e4a63ad37b1c86bbf1409: Status 404 returned error can't find the container with id 8985d94af986db7446a6967ff7b39f6220cefe152e4e4a63ad37b1c86bbf1409
Aug 05 18:18:18.714837 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.714756    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:18:18.807458 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.807425    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx"]
Aug 05 18:18:18.812299 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.812280    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx"
Aug 05 18:18:18.815013 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.814989    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-87a594\""
Aug 05 18:18:18.815124 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.815075    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-87a594-dockercfg-ck79c\""
Aug 05 18:18:18.820592 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.820567    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx"]
Aug 05 18:18:18.849730 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.849701    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:18:18.926696 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.926662    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjn6f\" (UniqueName: \"kubernetes.io/projected/74cb0bbc-52df-4802-a614-da5c835c623b-kube-api-access-pjn6f\") pod \"payload-processing-e2e-api-b-87a594-86db95c9fd-llltx\" (UID: \"74cb0bbc-52df-4802-a614-da5c835c623b\") " pod="openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx"
Aug 05 18:18:18.926846 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.926722    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74cb0bbc-52df-4802-a614-da5c835c623b-plugins-config-volume\") pod \"payload-processing-e2e-api-b-87a594-86db95c9fd-llltx\" (UID: \"74cb0bbc-52df-4802-a614-da5c835c623b\") " pod="openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx"
Aug 05 18:18:18.971429 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:18.971362    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb" event={"ID":"180d876b-8d08-4d92-afe3-de4359cadd42","Type":"ContainerStarted","Data":"8985d94af986db7446a6967ff7b39f6220cefe152e4e4a63ad37b1c86bbf1409"}
Aug 05 18:18:19.027978 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:19.027955    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjn6f\" (UniqueName: \"kubernetes.io/projected/74cb0bbc-52df-4802-a614-da5c835c623b-kube-api-access-pjn6f\") pod \"payload-processing-e2e-api-b-87a594-86db95c9fd-llltx\" (UID: \"74cb0bbc-52df-4802-a614-da5c835c623b\") " pod="openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx"
Aug 05 18:18:19.028118 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:19.028000    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74cb0bbc-52df-4802-a614-da5c835c623b-plugins-config-volume\") pod \"payload-processing-e2e-api-b-87a594-86db95c9fd-llltx\" (UID: \"74cb0bbc-52df-4802-a614-da5c835c623b\") " pod="openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx"
Aug 05 18:18:19.028706 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:19.028687    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74cb0bbc-52df-4802-a614-da5c835c623b-plugins-config-volume\") pod \"payload-processing-e2e-api-b-87a594-86db95c9fd-llltx\" (UID: \"74cb0bbc-52df-4802-a614-da5c835c623b\") " pod="openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx"
Aug 05 18:18:19.037057 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:19.037033    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjn6f\" (UniqueName: \"kubernetes.io/projected/74cb0bbc-52df-4802-a614-da5c835c623b-kube-api-access-pjn6f\") pod \"payload-processing-e2e-api-b-87a594-86db95c9fd-llltx\" (UID: \"74cb0bbc-52df-4802-a614-da5c835c623b\") " pod="openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx"
Aug 05 18:18:19.125814 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:19.125794    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx"
Aug 05 18:18:19.273711 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:19.273679    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx"]
Aug 05 18:18:19.275551 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:18:19.275529    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74cb0bbc_52df_4802_a614_da5c835c623b.slice/crio-e8a92ecf3b2edabd1c1971bbc1754b59f2f7f51058d68ae5f7bbc70a67373b6c WatchSource:0}: Error finding container e8a92ecf3b2edabd1c1971bbc1754b59f2f7f51058d68ae5f7bbc70a67373b6c: Status 404 returned error can't find the container with id e8a92ecf3b2edabd1c1971bbc1754b59f2f7f51058d68ae5f7bbc70a67373b6c
Aug 05 18:18:19.979787 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:19.979749    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb" event={"ID":"180d876b-8d08-4d92-afe3-de4359cadd42","Type":"ContainerStarted","Data":"42d1bf0fefc64cfafdbe686029c4758d24c8ad4d3ccf3ccc9535bf4b7b055fa9"}
Aug 05 18:18:19.979998 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:19.979821    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"
Aug 05 18:18:19.981323 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:19.981295    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx" event={"ID":"74cb0bbc-52df-4802-a614-da5c835c623b","Type":"ContainerStarted","Data":"c8e42a8dd58b8647c500f7db089a6a850b9c53df4c82d5b520195f3b95ce0f20"}
Aug 05 18:18:19.981431 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:19.981345    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx"
Aug 05 18:18:19.981431 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:19.981372    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx" event={"ID":"74cb0bbc-52df-4802-a614-da5c835c623b","Type":"ContainerStarted","Data":"e8a92ecf3b2edabd1c1971bbc1754b59f2f7f51058d68ae5f7bbc70a67373b6c"}
Aug 05 18:18:20.002205 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:20.002127    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb" podStartSLOduration=1.480672059 podStartE2EDuration="2.002110435s" totalImagesPullingTime="521.438376ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:18:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:18:20.00153376 +0000 UTC m=+2707.354208354" watchObservedRunningTime="2026-08-05 18:18:20.002110435 +0000 UTC m=+2707.354785032"
Aug 05 18:18:20.033795 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:20.033698    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx" podStartSLOduration=2.033682665 podStartE2EDuration="2.033682665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:18:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:18:20.033018402 +0000 UTC m=+2707.385692977" watchObservedRunningTime="2026-08-05 18:18:20.033682665 +0000 UTC m=+2707.386357260"
Aug 05 18:18:25.992684 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:25.992652    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"
Aug 05 18:18:30.989285 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:30.989255    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx"
Aug 05 18:18:47.140728 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:18:47.140693    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:05.468886 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.468847    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"]
Aug 05 18:19:05.470345 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.470082    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895" podUID="e5599c65-a852-434c-ba64-cfad5d82acd2" containerName="maas-api" containerID="cri-o://2d134d1eab6521caeedc103612a0ff7bf01fe62f44ff283e41197fd862f6b0b6" gracePeriod=30
Aug 05 18:19:05.495487 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.495449    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:05.540585 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.540539    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb"]
Aug 05 18:19:05.540840 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.540808    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb" podUID="5f75f09b-e111-4be0-b410-f45283a55b41" containerName="payload-processing" containerID="cri-o://e499e8629c688df9636808fca144abf2945e68e287d1495a8ad819cbc67378d1" gracePeriod=30
Aug 05 18:19:05.554955 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.554920    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6"]
Aug 05 18:19:05.555313 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.555281    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6" podUID="65972304-ea5f-40fe-a7f1-4e9e283a8f93" containerName="payload-pre-processing" containerID="cri-o://db85528eb4fc90a626bd71aea78036c42f5383c3241c7d6fe7b0ad29cce5ca41" gracePeriod=30
Aug 05 18:19:05.595624 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.595583    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:05.694703 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.694342    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:05.771027 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.770997    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:05.858981 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.858951    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb"
Aug 05 18:19:05.863116 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.863085    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"
Aug 05 18:19:05.867374 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.867344    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6"
Aug 05 18:19:05.963740 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.963703    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgjdj\" (UniqueName: \"kubernetes.io/projected/5f75f09b-e111-4be0-b410-f45283a55b41-kube-api-access-lgjdj\") pod \"5f75f09b-e111-4be0-b410-f45283a55b41\" (UID: \"5f75f09b-e111-4be0-b410-f45283a55b41\") "
Aug 05 18:19:05.963740 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.963763    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5599c65-a852-434c-ba64-cfad5d82acd2-maas-api-tls\") pod \"e5599c65-a852-434c-ba64-cfad5d82acd2\" (UID: \"e5599c65-a852-434c-ba64-cfad5d82acd2\") "
Aug 05 18:19:05.964027 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.963817    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtzkb\" (UniqueName: \"kubernetes.io/projected/65972304-ea5f-40fe-a7f1-4e9e283a8f93-kube-api-access-dtzkb\") pod \"65972304-ea5f-40fe-a7f1-4e9e283a8f93\" (UID: \"65972304-ea5f-40fe-a7f1-4e9e283a8f93\") "
Aug 05 18:19:05.964027 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.963847    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5599c65-a852-434c-ba64-cfad5d82acd2-openshift-service-ca\") pod \"e5599c65-a852-434c-ba64-cfad5d82acd2\" (UID: \"e5599c65-a852-434c-ba64-cfad5d82acd2\") "
Aug 05 18:19:05.964027 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.963879    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f75f09b-e111-4be0-b410-f45283a55b41-plugins-config-volume\") pod \"5f75f09b-e111-4be0-b410-f45283a55b41\" (UID: \"5f75f09b-e111-4be0-b410-f45283a55b41\") "
Aug 05 18:19:05.964027 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.963916    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65972304-ea5f-40fe-a7f1-4e9e283a8f93-plugins-config-volume\") pod \"65972304-ea5f-40fe-a7f1-4e9e283a8f93\" (UID: \"65972304-ea5f-40fe-a7f1-4e9e283a8f93\") "
Aug 05 18:19:05.964027 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.963988    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8spn\" (UniqueName: \"kubernetes.io/projected/e5599c65-a852-434c-ba64-cfad5d82acd2-kube-api-access-l8spn\") pod \"e5599c65-a852-434c-ba64-cfad5d82acd2\" (UID: \"e5599c65-a852-434c-ba64-cfad5d82acd2\") "
Aug 05 18:19:05.964321 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.964278    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5599c65-a852-434c-ba64-cfad5d82acd2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e5599c65-a852-434c-ba64-cfad5d82acd2" (UID: "e5599c65-a852-434c-ba64-cfad5d82acd2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:19:05.964768 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.964740    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65972304-ea5f-40fe-a7f1-4e9e283a8f93-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "65972304-ea5f-40fe-a7f1-4e9e283a8f93" (UID: "65972304-ea5f-40fe-a7f1-4e9e283a8f93"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:19:05.964902 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.964796    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f75f09b-e111-4be0-b410-f45283a55b41-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5f75f09b-e111-4be0-b410-f45283a55b41" (UID: "5f75f09b-e111-4be0-b410-f45283a55b41"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:19:05.966868 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.966832    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5599c65-a852-434c-ba64-cfad5d82acd2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e5599c65-a852-434c-ba64-cfad5d82acd2" (UID: "e5599c65-a852-434c-ba64-cfad5d82acd2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:19:05.966976 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.966915    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65972304-ea5f-40fe-a7f1-4e9e283a8f93-kube-api-access-dtzkb" (OuterVolumeSpecName: "kube-api-access-dtzkb") pod "65972304-ea5f-40fe-a7f1-4e9e283a8f93" (UID: "65972304-ea5f-40fe-a7f1-4e9e283a8f93"). InnerVolumeSpecName "kube-api-access-dtzkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:19:05.966976 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.966934    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5599c65-a852-434c-ba64-cfad5d82acd2-kube-api-access-l8spn" (OuterVolumeSpecName: "kube-api-access-l8spn") pod "e5599c65-a852-434c-ba64-cfad5d82acd2" (UID: "e5599c65-a852-434c-ba64-cfad5d82acd2"). InnerVolumeSpecName "kube-api-access-l8spn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:19:05.967065 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:05.967050    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f75f09b-e111-4be0-b410-f45283a55b41-kube-api-access-lgjdj" (OuterVolumeSpecName: "kube-api-access-lgjdj") pod "5f75f09b-e111-4be0-b410-f45283a55b41" (UID: "5f75f09b-e111-4be0-b410-f45283a55b41"). InnerVolumeSpecName "kube-api-access-lgjdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:19:06.060533 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.060456    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:06.064899 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.064881    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65972304-ea5f-40fe-a7f1-4e9e283a8f93-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:19:06.064977 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.064903    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8spn\" (UniqueName: \"kubernetes.io/projected/e5599c65-a852-434c-ba64-cfad5d82acd2-kube-api-access-l8spn\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:19:06.064977 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.064912    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgjdj\" (UniqueName: \"kubernetes.io/projected/5f75f09b-e111-4be0-b410-f45283a55b41-kube-api-access-lgjdj\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:19:06.064977 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.064920    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5599c65-a852-434c-ba64-cfad5d82acd2-maas-api-tls\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:19:06.064977 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.064929    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtzkb\" (UniqueName: \"kubernetes.io/projected/65972304-ea5f-40fe-a7f1-4e9e283a8f93-kube-api-access-dtzkb\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:19:06.064977 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.064938    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5599c65-a852-434c-ba64-cfad5d82acd2-openshift-service-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:19:06.064977 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.064946    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f75f09b-e111-4be0-b410-f45283a55b41-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:19:06.225673 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.225640    2569 generic.go:356] "Generic (PLEG): container finished" podID="65972304-ea5f-40fe-a7f1-4e9e283a8f93" containerID="db85528eb4fc90a626bd71aea78036c42f5383c3241c7d6fe7b0ad29cce5ca41" exitCode=0
Aug 05 18:19:06.225857 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.225688    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6"
Aug 05 18:19:06.225857 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.225730    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6" event={"ID":"65972304-ea5f-40fe-a7f1-4e9e283a8f93","Type":"ContainerDied","Data":"db85528eb4fc90a626bd71aea78036c42f5383c3241c7d6fe7b0ad29cce5ca41"}
Aug 05 18:19:06.225857 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.225787    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6" event={"ID":"65972304-ea5f-40fe-a7f1-4e9e283a8f93","Type":"ContainerDied","Data":"556ac62112985d27e3afad0da70f777a3741d24132fe8875298853d14a2205bc"}
Aug 05 18:19:06.225857 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.225812    2569 scope.go:117] "RemoveContainer" containerID="db85528eb4fc90a626bd71aea78036c42f5383c3241c7d6fe7b0ad29cce5ca41"
Aug 05 18:19:06.227012 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.226992    2569 generic.go:356] "Generic (PLEG): container finished" podID="e5599c65-a852-434c-ba64-cfad5d82acd2" containerID="2d134d1eab6521caeedc103612a0ff7bf01fe62f44ff283e41197fd862f6b0b6" exitCode=0
Aug 05 18:19:06.227084 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.227040    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"
Aug 05 18:19:06.227146 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.227079    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895" event={"ID":"e5599c65-a852-434c-ba64-cfad5d82acd2","Type":"ContainerDied","Data":"2d134d1eab6521caeedc103612a0ff7bf01fe62f44ff283e41197fd862f6b0b6"}
Aug 05 18:19:06.227146 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.227128    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895" event={"ID":"e5599c65-a852-434c-ba64-cfad5d82acd2","Type":"ContainerDied","Data":"3d98963e4665299d14d2324799f787f135dca04011f0079a40fab12d7e02e885"}
Aug 05 18:19:06.228881 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.228792    2569 generic.go:356] "Generic (PLEG): container finished" podID="5f75f09b-e111-4be0-b410-f45283a55b41" containerID="e499e8629c688df9636808fca144abf2945e68e287d1495a8ad819cbc67378d1" exitCode=0
Aug 05 18:19:06.228881 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.228843    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb"
Aug 05 18:19:06.228881 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.228843    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb" event={"ID":"5f75f09b-e111-4be0-b410-f45283a55b41","Type":"ContainerDied","Data":"e499e8629c688df9636808fca144abf2945e68e287d1495a8ad819cbc67378d1"}
Aug 05 18:19:06.229118 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.228890    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb" event={"ID":"5f75f09b-e111-4be0-b410-f45283a55b41","Type":"ContainerDied","Data":"73407f0d8927285cef4d6772f7129342250da36634ab7eb8640907346c0c0556"}
Aug 05 18:19:06.236245 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.236210    2569 scope.go:117] "RemoveContainer" containerID="db85528eb4fc90a626bd71aea78036c42f5383c3241c7d6fe7b0ad29cce5ca41"
Aug 05 18:19:06.236511 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:19:06.236490    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db85528eb4fc90a626bd71aea78036c42f5383c3241c7d6fe7b0ad29cce5ca41\": container with ID starting with db85528eb4fc90a626bd71aea78036c42f5383c3241c7d6fe7b0ad29cce5ca41 not found: ID does not exist" containerID="db85528eb4fc90a626bd71aea78036c42f5383c3241c7d6fe7b0ad29cce5ca41"
Aug 05 18:19:06.236583 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.236524    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db85528eb4fc90a626bd71aea78036c42f5383c3241c7d6fe7b0ad29cce5ca41"} err="failed to get container status \"db85528eb4fc90a626bd71aea78036c42f5383c3241c7d6fe7b0ad29cce5ca41\": rpc error: code = NotFound desc = could not find container \"db85528eb4fc90a626bd71aea78036c42f5383c3241c7d6fe7b0ad29cce5ca41\": container with ID starting with db85528eb4fc90a626bd71aea78036c42f5383c3241c7d6fe7b0ad29cce5ca41 not found: ID does not exist"
Aug 05 18:19:06.236583 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.236548    2569 scope.go:117] "RemoveContainer" containerID="2d134d1eab6521caeedc103612a0ff7bf01fe62f44ff283e41197fd862f6b0b6"
Aug 05 18:19:06.245133 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.245118    2569 scope.go:117] "RemoveContainer" containerID="2d134d1eab6521caeedc103612a0ff7bf01fe62f44ff283e41197fd862f6b0b6"
Aug 05 18:19:06.245406 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:19:06.245390    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d134d1eab6521caeedc103612a0ff7bf01fe62f44ff283e41197fd862f6b0b6\": container with ID starting with 2d134d1eab6521caeedc103612a0ff7bf01fe62f44ff283e41197fd862f6b0b6 not found: ID does not exist" containerID="2d134d1eab6521caeedc103612a0ff7bf01fe62f44ff283e41197fd862f6b0b6"
Aug 05 18:19:06.245464 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.245412    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d134d1eab6521caeedc103612a0ff7bf01fe62f44ff283e41197fd862f6b0b6"} err="failed to get container status \"2d134d1eab6521caeedc103612a0ff7bf01fe62f44ff283e41197fd862f6b0b6\": rpc error: code = NotFound desc = could not find container \"2d134d1eab6521caeedc103612a0ff7bf01fe62f44ff283e41197fd862f6b0b6\": container with ID starting with 2d134d1eab6521caeedc103612a0ff7bf01fe62f44ff283e41197fd862f6b0b6 not found: ID does not exist"
Aug 05 18:19:06.245464 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.245427    2569 scope.go:117] "RemoveContainer" containerID="e499e8629c688df9636808fca144abf2945e68e287d1495a8ad819cbc67378d1"
Aug 05 18:19:06.252627 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.252606    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6"]
Aug 05 18:19:06.253565 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.253550    2569 scope.go:117] "RemoveContainer" containerID="e499e8629c688df9636808fca144abf2945e68e287d1495a8ad819cbc67378d1"
Aug 05 18:19:06.253815 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:19:06.253798    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e499e8629c688df9636808fca144abf2945e68e287d1495a8ad819cbc67378d1\": container with ID starting with e499e8629c688df9636808fca144abf2945e68e287d1495a8ad819cbc67378d1 not found: ID does not exist" containerID="e499e8629c688df9636808fca144abf2945e68e287d1495a8ad819cbc67378d1"
Aug 05 18:19:06.253877 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.253821    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e499e8629c688df9636808fca144abf2945e68e287d1495a8ad819cbc67378d1"} err="failed to get container status \"e499e8629c688df9636808fca144abf2945e68e287d1495a8ad819cbc67378d1\": rpc error: code = NotFound desc = could not find container \"e499e8629c688df9636808fca144abf2945e68e287d1495a8ad819cbc67378d1\": container with ID starting with e499e8629c688df9636808fca144abf2945e68e287d1495a8ad819cbc67378d1 not found: ID does not exist"
Aug 05 18:19:06.257370 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.257351    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-acb45a-74d6dc57fb-52nj6"]
Aug 05 18:19:06.270284 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.270260    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb"]
Aug 05 18:19:06.274241 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.274222    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-acb45a-5d7db6c748-dd7nb"]
Aug 05 18:19:06.286891 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.286867    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"]
Aug 05 18:19:06.294434 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:06.294414    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-acb45a-5b55d98b8c-7t895"]
Aug 05 18:19:07.151906 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:07.151867    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f75f09b-e111-4be0-b410-f45283a55b41" path="/var/lib/kubelet/pods/5f75f09b-e111-4be0-b410-f45283a55b41/volumes"
Aug 05 18:19:07.152334 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:07.152318    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65972304-ea5f-40fe-a7f1-4e9e283a8f93" path="/var/lib/kubelet/pods/65972304-ea5f-40fe-a7f1-4e9e283a8f93/volumes"
Aug 05 18:19:07.152728 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:07.152714    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5599c65-a852-434c-ba64-cfad5d82acd2" path="/var/lib/kubelet/pods/e5599c65-a852-434c-ba64-cfad5d82acd2/volumes"
Aug 05 18:19:12.201939 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:12.201905    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:12.312422 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:12.312389    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:12.527829 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:12.527755    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:12.704705 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:12.704670    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:28.348063 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.348027    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"]
Aug 05 18:19:28.349124 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.349087    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb" podUID="180d876b-8d08-4d92-afe3-de4359cadd42" containerName="maas-api" containerID="cri-o://42d1bf0fefc64cfafdbe686029c4758d24c8ad4d3ccf3ccc9535bf4b7b055fa9" gracePeriod=30
Aug 05 18:19:28.373476 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.373453    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:28.398628 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.398604    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx"]
Aug 05 18:19:28.398833 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.398810    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx" podUID="74cb0bbc-52df-4802-a614-da5c835c623b" containerName="payload-processing" containerID="cri-o://c8e42a8dd58b8647c500f7db089a6a850b9c53df4c82d5b520195f3b95ce0f20" gracePeriod=30
Aug 05 18:19:28.446115 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.446084    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:28.541551 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.541519    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:28.601122 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.601051    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:28.668058 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.668027    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"
Aug 05 18:19:28.671865 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.671847    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx"
Aug 05 18:19:28.763777 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.763749    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/180d876b-8d08-4d92-afe3-de4359cadd42-maas-api-tls\") pod \"180d876b-8d08-4d92-afe3-de4359cadd42\" (UID: \"180d876b-8d08-4d92-afe3-de4359cadd42\") "
Aug 05 18:19:28.763966 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.763797    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/180d876b-8d08-4d92-afe3-de4359cadd42-openshift-service-ca\") pod \"180d876b-8d08-4d92-afe3-de4359cadd42\" (UID: \"180d876b-8d08-4d92-afe3-de4359cadd42\") "
Aug 05 18:19:28.763966 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.763823    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jg2vg\" (UniqueName: \"kubernetes.io/projected/180d876b-8d08-4d92-afe3-de4359cadd42-kube-api-access-jg2vg\") pod \"180d876b-8d08-4d92-afe3-de4359cadd42\" (UID: \"180d876b-8d08-4d92-afe3-de4359cadd42\") "
Aug 05 18:19:28.764217 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.764173    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/180d876b-8d08-4d92-afe3-de4359cadd42-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "180d876b-8d08-4d92-afe3-de4359cadd42" (UID: "180d876b-8d08-4d92-afe3-de4359cadd42"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:19:28.766627 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.766591    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/180d876b-8d08-4d92-afe3-de4359cadd42-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "180d876b-8d08-4d92-afe3-de4359cadd42" (UID: "180d876b-8d08-4d92-afe3-de4359cadd42"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:19:28.766627 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.766606    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/180d876b-8d08-4d92-afe3-de4359cadd42-kube-api-access-jg2vg" (OuterVolumeSpecName: "kube-api-access-jg2vg") pod "180d876b-8d08-4d92-afe3-de4359cadd42" (UID: "180d876b-8d08-4d92-afe3-de4359cadd42"). InnerVolumeSpecName "kube-api-access-jg2vg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:19:28.865340 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.865251    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjn6f\" (UniqueName: \"kubernetes.io/projected/74cb0bbc-52df-4802-a614-da5c835c623b-kube-api-access-pjn6f\") pod \"74cb0bbc-52df-4802-a614-da5c835c623b\" (UID: \"74cb0bbc-52df-4802-a614-da5c835c623b\") "
Aug 05 18:19:28.865499 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.865368    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74cb0bbc-52df-4802-a614-da5c835c623b-plugins-config-volume\") pod \"74cb0bbc-52df-4802-a614-da5c835c623b\" (UID: \"74cb0bbc-52df-4802-a614-da5c835c623b\") "
Aug 05 18:19:28.865547 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.865540    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/180d876b-8d08-4d92-afe3-de4359cadd42-openshift-service-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:19:28.865585 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.865551    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jg2vg\" (UniqueName: \"kubernetes.io/projected/180d876b-8d08-4d92-afe3-de4359cadd42-kube-api-access-jg2vg\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:19:28.865585 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.865560    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/180d876b-8d08-4d92-afe3-de4359cadd42-maas-api-tls\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:19:28.865745 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.865721    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74cb0bbc-52df-4802-a614-da5c835c623b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "74cb0bbc-52df-4802-a614-da5c835c623b" (UID: "74cb0bbc-52df-4802-a614-da5c835c623b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:19:28.867560 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.867541    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74cb0bbc-52df-4802-a614-da5c835c623b-kube-api-access-pjn6f" (OuterVolumeSpecName: "kube-api-access-pjn6f") pod "74cb0bbc-52df-4802-a614-da5c835c623b" (UID: "74cb0bbc-52df-4802-a614-da5c835c623b"). InnerVolumeSpecName "kube-api-access-pjn6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:19:28.939451 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.939418    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:28.966640 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.966610    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74cb0bbc-52df-4802-a614-da5c835c623b-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:19:28.966640 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:28.966636    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjn6f\" (UniqueName: \"kubernetes.io/projected/74cb0bbc-52df-4802-a614-da5c835c623b-kube-api-access-pjn6f\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:19:29.353780 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:29.353745    2569 generic.go:356] "Generic (PLEG): container finished" podID="74cb0bbc-52df-4802-a614-da5c835c623b" containerID="c8e42a8dd58b8647c500f7db089a6a850b9c53df4c82d5b520195f3b95ce0f20" exitCode=0
Aug 05 18:19:29.354225 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:29.353788    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx"
Aug 05 18:19:29.354225 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:29.353828    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx" event={"ID":"74cb0bbc-52df-4802-a614-da5c835c623b","Type":"ContainerDied","Data":"c8e42a8dd58b8647c500f7db089a6a850b9c53df4c82d5b520195f3b95ce0f20"}
Aug 05 18:19:29.354225 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:29.353881    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx" event={"ID":"74cb0bbc-52df-4802-a614-da5c835c623b","Type":"ContainerDied","Data":"e8a92ecf3b2edabd1c1971bbc1754b59f2f7f51058d68ae5f7bbc70a67373b6c"}
Aug 05 18:19:29.354225 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:29.353913    2569 scope.go:117] "RemoveContainer" containerID="c8e42a8dd58b8647c500f7db089a6a850b9c53df4c82d5b520195f3b95ce0f20"
Aug 05 18:19:29.355369 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:29.355344    2569 generic.go:356] "Generic (PLEG): container finished" podID="180d876b-8d08-4d92-afe3-de4359cadd42" containerID="42d1bf0fefc64cfafdbe686029c4758d24c8ad4d3ccf3ccc9535bf4b7b055fa9" exitCode=0
Aug 05 18:19:29.355471 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:29.355390    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"
Aug 05 18:19:29.355471 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:29.355410    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb" event={"ID":"180d876b-8d08-4d92-afe3-de4359cadd42","Type":"ContainerDied","Data":"42d1bf0fefc64cfafdbe686029c4758d24c8ad4d3ccf3ccc9535bf4b7b055fa9"}
Aug 05 18:19:29.355471 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:29.355451    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb" event={"ID":"180d876b-8d08-4d92-afe3-de4359cadd42","Type":"ContainerDied","Data":"8985d94af986db7446a6967ff7b39f6220cefe152e4e4a63ad37b1c86bbf1409"}
Aug 05 18:19:29.370076 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:29.370059    2569 scope.go:117] "RemoveContainer" containerID="c8e42a8dd58b8647c500f7db089a6a850b9c53df4c82d5b520195f3b95ce0f20"
Aug 05 18:19:29.370387 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:19:29.370367    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8e42a8dd58b8647c500f7db089a6a850b9c53df4c82d5b520195f3b95ce0f20\": container with ID starting with c8e42a8dd58b8647c500f7db089a6a850b9c53df4c82d5b520195f3b95ce0f20 not found: ID does not exist" containerID="c8e42a8dd58b8647c500f7db089a6a850b9c53df4c82d5b520195f3b95ce0f20"
Aug 05 18:19:29.370503 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:29.370395    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8e42a8dd58b8647c500f7db089a6a850b9c53df4c82d5b520195f3b95ce0f20"} err="failed to get container status \"c8e42a8dd58b8647c500f7db089a6a850b9c53df4c82d5b520195f3b95ce0f20\": rpc error: code = NotFound desc = could not find container \"c8e42a8dd58b8647c500f7db089a6a850b9c53df4c82d5b520195f3b95ce0f20\": container with ID starting with c8e42a8dd58b8647c500f7db089a6a850b9c53df4c82d5b520195f3b95ce0f20 not found: ID does not exist"
Aug 05 18:19:29.370503 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:29.370412    2569 scope.go:117] "RemoveContainer" containerID="42d1bf0fefc64cfafdbe686029c4758d24c8ad4d3ccf3ccc9535bf4b7b055fa9"
Aug 05 18:19:29.378583 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:29.378567    2569 scope.go:117] "RemoveContainer" containerID="42d1bf0fefc64cfafdbe686029c4758d24c8ad4d3ccf3ccc9535bf4b7b055fa9"
Aug 05 18:19:29.378842 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:19:29.378824    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42d1bf0fefc64cfafdbe686029c4758d24c8ad4d3ccf3ccc9535bf4b7b055fa9\": container with ID starting with 42d1bf0fefc64cfafdbe686029c4758d24c8ad4d3ccf3ccc9535bf4b7b055fa9 not found: ID does not exist" containerID="42d1bf0fefc64cfafdbe686029c4758d24c8ad4d3ccf3ccc9535bf4b7b055fa9"
Aug 05 18:19:29.378927 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:29.378849    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42d1bf0fefc64cfafdbe686029c4758d24c8ad4d3ccf3ccc9535bf4b7b055fa9"} err="failed to get container status \"42d1bf0fefc64cfafdbe686029c4758d24c8ad4d3ccf3ccc9535bf4b7b055fa9\": rpc error: code = NotFound desc = could not find container \"42d1bf0fefc64cfafdbe686029c4758d24c8ad4d3ccf3ccc9535bf4b7b055fa9\": container with ID starting with 42d1bf0fefc64cfafdbe686029c4758d24c8ad4d3ccf3ccc9535bf4b7b055fa9 not found: ID does not exist"
Aug 05 18:19:29.380227 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:29.380208    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx"]
Aug 05 18:19:29.387036 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:29.387015    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-87a594-86db95c9fd-llltx"]
Aug 05 18:19:29.399911 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:29.399885    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"]
Aug 05 18:19:29.404399 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:29.404378    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-87a594-84b48744d6-2psrb"]
Aug 05 18:19:31.151299 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:31.151265    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="180d876b-8d08-4d92-afe3-de4359cadd42" path="/var/lib/kubelet/pods/180d876b-8d08-4d92-afe3-de4359cadd42/volumes"
Aug 05 18:19:31.151689 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:31.151680    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74cb0bbc-52df-4802-a614-da5c835c623b" path="/var/lib/kubelet/pods/74cb0bbc-52df-4802-a614-da5c835c623b/volumes"
Aug 05 18:19:35.174070 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:35.173995    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:35.233942 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:35.233909    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:35.444719 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:35.444640    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:35.657499 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:35.657468    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:35.962293 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:35.962258    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:36.286150 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:36.286065    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:36.525309 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:36.525280    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:39.359547 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:39.359511    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:39.457858 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:39.457829    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:40.460054 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.460015    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"]
Aug 05 18:19:40.460494 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.460478    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65972304-ea5f-40fe-a7f1-4e9e283a8f93" containerName="payload-pre-processing"
Aug 05 18:19:40.460566 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.460496    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="65972304-ea5f-40fe-a7f1-4e9e283a8f93" containerName="payload-pre-processing"
Aug 05 18:19:40.460566 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.460506    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f75f09b-e111-4be0-b410-f45283a55b41" containerName="payload-processing"
Aug 05 18:19:40.460566 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.460511    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f75f09b-e111-4be0-b410-f45283a55b41" containerName="payload-processing"
Aug 05 18:19:40.460566 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.460522    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5599c65-a852-434c-ba64-cfad5d82acd2" containerName="maas-api"
Aug 05 18:19:40.460566 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.460527    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5599c65-a852-434c-ba64-cfad5d82acd2" containerName="maas-api"
Aug 05 18:19:40.460566 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.460543    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74cb0bbc-52df-4802-a614-da5c835c623b" containerName="payload-processing"
Aug 05 18:19:40.460566 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.460548    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="74cb0bbc-52df-4802-a614-da5c835c623b" containerName="payload-processing"
Aug 05 18:19:40.460566 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.460555    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="180d876b-8d08-4d92-afe3-de4359cadd42" containerName="maas-api"
Aug 05 18:19:40.460566 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.460561    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="180d876b-8d08-4d92-afe3-de4359cadd42" containerName="maas-api"
Aug 05 18:19:40.460924 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.460618    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5599c65-a852-434c-ba64-cfad5d82acd2" containerName="maas-api"
Aug 05 18:19:40.460924 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.460625    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="74cb0bbc-52df-4802-a614-da5c835c623b" containerName="payload-processing"
Aug 05 18:19:40.460924 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.460631    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="65972304-ea5f-40fe-a7f1-4e9e283a8f93" containerName="payload-pre-processing"
Aug 05 18:19:40.460924 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.460639    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="180d876b-8d08-4d92-afe3-de4359cadd42" containerName="maas-api"
Aug 05 18:19:40.460924 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.460644    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f75f09b-e111-4be0-b410-f45283a55b41" containerName="payload-processing"
Aug 05 18:19:40.463774 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.463751    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"
Aug 05 18:19:40.466581 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.466555    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-70b4a4-serving-cert\""
Aug 05 18:19:40.466758 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.466641    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jdhhr\""
Aug 05 18:19:40.477135 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.477103    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"]
Aug 05 18:19:40.490124 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.490026    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:40.570021 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.569987    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7drff\" (UniqueName: \"kubernetes.io/projected/77e76780-cf13-4569-ba5a-43bcb5409418-kube-api-access-7drff\") pod \"maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj\" (UID: \"77e76780-cf13-4569-ba5a-43bcb5409418\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"
Aug 05 18:19:40.570234 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.570036    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77e76780-cf13-4569-ba5a-43bcb5409418-maas-api-tls\") pod \"maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj\" (UID: \"77e76780-cf13-4569-ba5a-43bcb5409418\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"
Aug 05 18:19:40.570234 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.570068    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77e76780-cf13-4569-ba5a-43bcb5409418-openshift-service-ca\") pod \"maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj\" (UID: \"77e76780-cf13-4569-ba5a-43bcb5409418\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"
Aug 05 18:19:40.671644 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.671594    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7drff\" (UniqueName: \"kubernetes.io/projected/77e76780-cf13-4569-ba5a-43bcb5409418-kube-api-access-7drff\") pod \"maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj\" (UID: \"77e76780-cf13-4569-ba5a-43bcb5409418\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"
Aug 05 18:19:40.671834 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.671684    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77e76780-cf13-4569-ba5a-43bcb5409418-maas-api-tls\") pod \"maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj\" (UID: \"77e76780-cf13-4569-ba5a-43bcb5409418\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"
Aug 05 18:19:40.671834 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.671721    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77e76780-cf13-4569-ba5a-43bcb5409418-openshift-service-ca\") pod \"maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj\" (UID: \"77e76780-cf13-4569-ba5a-43bcb5409418\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"
Aug 05 18:19:40.671834 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:19:40.671821    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-serving-cert: secret "maas-api-e2e-shared-a-70b4a4-serving-cert" not found
Aug 05 18:19:40.672024 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:19:40.671887    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77e76780-cf13-4569-ba5a-43bcb5409418-maas-api-tls podName:77e76780-cf13-4569-ba5a-43bcb5409418 nodeName:}" failed. No retries permitted until 2026-08-05 18:19:41.171870341 +0000 UTC m=+2788.524544914 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/77e76780-cf13-4569-ba5a-43bcb5409418-maas-api-tls") pod "maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj" (UID: "77e76780-cf13-4569-ba5a-43bcb5409418") : secret "maas-api-e2e-shared-a-70b4a4-serving-cert" not found
Aug 05 18:19:40.672473 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.672451    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77e76780-cf13-4569-ba5a-43bcb5409418-openshift-service-ca\") pod \"maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj\" (UID: \"77e76780-cf13-4569-ba5a-43bcb5409418\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"
Aug 05 18:19:40.685540 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.685506    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7drff\" (UniqueName: \"kubernetes.io/projected/77e76780-cf13-4569-ba5a-43bcb5409418-kube-api-access-7drff\") pod \"maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj\" (UID: \"77e76780-cf13-4569-ba5a-43bcb5409418\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"
Aug 05 18:19:40.742002 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:40.741879    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:41.098791 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.098759    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:41.177570 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.177534    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77e76780-cf13-4569-ba5a-43bcb5409418-maas-api-tls\") pod \"maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj\" (UID: \"77e76780-cf13-4569-ba5a-43bcb5409418\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"
Aug 05 18:19:41.180015 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.179989    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77e76780-cf13-4569-ba5a-43bcb5409418-maas-api-tls\") pod \"maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj\" (UID: \"77e76780-cf13-4569-ba5a-43bcb5409418\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"
Aug 05 18:19:41.200136 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.200105    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:41.217309 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.217282    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd"]
Aug 05 18:19:41.221003 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.220976    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd"
Aug 05 18:19:41.223968 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.223934    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-70b4a4\""
Aug 05 18:19:41.224139 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.224126    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-70b4a4-dockercfg-5ts86\""
Aug 05 18:19:41.237855 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.237826    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd"]
Aug 05 18:19:41.241164 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.241135    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr"]
Aug 05 18:19:41.245777 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.245755    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr"
Aug 05 18:19:41.252471 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.252444    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr"]
Aug 05 18:19:41.278610 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.278580    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fbh2\" (UniqueName: \"kubernetes.io/projected/f2473b44-6051-4d8e-b17a-349d85441ae6-kube-api-access-7fbh2\") pod \"payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd\" (UID: \"f2473b44-6051-4d8e-b17a-349d85441ae6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd"
Aug 05 18:19:41.278797 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.278639    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e746138a-5050-4ec5-98db-304839bffc5c-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr\" (UID: \"e746138a-5050-4ec5-98db-304839bffc5c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr"
Aug 05 18:19:41.278797 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.278700    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2473b44-6051-4d8e-b17a-349d85441ae6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd\" (UID: \"f2473b44-6051-4d8e-b17a-349d85441ae6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd"
Aug 05 18:19:41.278797 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.278740    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvggr\" (UniqueName: \"kubernetes.io/projected/e746138a-5050-4ec5-98db-304839bffc5c-kube-api-access-gvggr\") pod \"payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr\" (UID: \"e746138a-5050-4ec5-98db-304839bffc5c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr"
Aug 05 18:19:41.374983 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.374875    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"
Aug 05 18:19:41.380374 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.380345    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fbh2\" (UniqueName: \"kubernetes.io/projected/f2473b44-6051-4d8e-b17a-349d85441ae6-kube-api-access-7fbh2\") pod \"payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd\" (UID: \"f2473b44-6051-4d8e-b17a-349d85441ae6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd"
Aug 05 18:19:41.380473 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.380407    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e746138a-5050-4ec5-98db-304839bffc5c-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr\" (UID: \"e746138a-5050-4ec5-98db-304839bffc5c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr"
Aug 05 18:19:41.380473 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.380433    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2473b44-6051-4d8e-b17a-349d85441ae6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd\" (UID: \"f2473b44-6051-4d8e-b17a-349d85441ae6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd"
Aug 05 18:19:41.380473 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.380455    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvggr\" (UniqueName: \"kubernetes.io/projected/e746138a-5050-4ec5-98db-304839bffc5c-kube-api-access-gvggr\") pod \"payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr\" (UID: \"e746138a-5050-4ec5-98db-304839bffc5c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr"
Aug 05 18:19:41.381200 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.381152    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2473b44-6051-4d8e-b17a-349d85441ae6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd\" (UID: \"f2473b44-6051-4d8e-b17a-349d85441ae6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd"
Aug 05 18:19:41.381305 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.381255    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e746138a-5050-4ec5-98db-304839bffc5c-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr\" (UID: \"e746138a-5050-4ec5-98db-304839bffc5c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr"
Aug 05 18:19:41.389592 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.389564    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fbh2\" (UniqueName: \"kubernetes.io/projected/f2473b44-6051-4d8e-b17a-349d85441ae6-kube-api-access-7fbh2\") pod \"payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd\" (UID: \"f2473b44-6051-4d8e-b17a-349d85441ae6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd"
Aug 05 18:19:41.389592 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.389595    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvggr\" (UniqueName: \"kubernetes.io/projected/e746138a-5050-4ec5-98db-304839bffc5c-kube-api-access-gvggr\") pod \"payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr\" (UID: \"e746138a-5050-4ec5-98db-304839bffc5c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr"
Aug 05 18:19:41.534542 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.534509    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"]
Aug 05 18:19:41.535492 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.535464    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd"
Aug 05 18:19:41.536864 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:19:41.536840    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77e76780_cf13_4569_ba5a_43bcb5409418.slice/crio-6d046e296877926c93c5a9b43cf971f0f4b3806b19599bb069073dd2015d9b27 WatchSource:0}: Error finding container 6d046e296877926c93c5a9b43cf971f0f4b3806b19599bb069073dd2015d9b27: Status 404 returned error can't find the container with id 6d046e296877926c93c5a9b43cf971f0f4b3806b19599bb069073dd2015d9b27
Aug 05 18:19:41.561673 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.561642    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr"
Aug 05 18:19:41.705595 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.705567    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd"]
Aug 05 18:19:41.709192 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:19:41.709131    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2473b44_6051_4d8e_b17a_349d85441ae6.slice/crio-408c240a3c29d790e4416548dbc56c92cbeb63dd86f03d821c8067df10807e49 WatchSource:0}: Error finding container 408c240a3c29d790e4416548dbc56c92cbeb63dd86f03d821c8067df10807e49: Status 404 returned error can't find the container with id 408c240a3c29d790e4416548dbc56c92cbeb63dd86f03d821c8067df10807e49
Aug 05 18:19:41.730568 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:41.730540    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr"]
Aug 05 18:19:41.734285 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:19:41.734250    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode746138a_5050_4ec5_98db_304839bffc5c.slice/crio-584d20b978dbb731dfc664f078fc48a99d773d35cd932269ed3ebc69ee8d531f WatchSource:0}: Error finding container 584d20b978dbb731dfc664f078fc48a99d773d35cd932269ed3ebc69ee8d531f: Status 404 returned error can't find the container with id 584d20b978dbb731dfc664f078fc48a99d773d35cd932269ed3ebc69ee8d531f
Aug 05 18:19:42.426478 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:42.426441    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd" event={"ID":"f2473b44-6051-4d8e-b17a-349d85441ae6","Type":"ContainerStarted","Data":"c08fdeb715ebc17b9cd01bf38ca02de7d89e86fb750ea78c81255486242a41b8"}
Aug 05 18:19:42.426688 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:42.426500    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd" event={"ID":"f2473b44-6051-4d8e-b17a-349d85441ae6","Type":"ContainerStarted","Data":"408c240a3c29d790e4416548dbc56c92cbeb63dd86f03d821c8067df10807e49"}
Aug 05 18:19:42.426688 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:42.426595    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd"
Aug 05 18:19:42.428192 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:42.428142    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj" event={"ID":"77e76780-cf13-4569-ba5a-43bcb5409418","Type":"ContainerStarted","Data":"245d816a71abf4f7f82b1cf0c6363a4ce016f32729320ec8c5678b56fa33e3fa"}
Aug 05 18:19:42.428333 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:42.428209    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj" event={"ID":"77e76780-cf13-4569-ba5a-43bcb5409418","Type":"ContainerStarted","Data":"6d046e296877926c93c5a9b43cf971f0f4b3806b19599bb069073dd2015d9b27"}
Aug 05 18:19:42.428333 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:42.428239    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"
Aug 05 18:19:42.429787 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:42.429765    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr" event={"ID":"e746138a-5050-4ec5-98db-304839bffc5c","Type":"ContainerStarted","Data":"81ad8506828f2ef2f34e7bb36bb31b8302538fd998ee24861d50e8e43cb1766d"}
Aug 05 18:19:42.429936 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:42.429811    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr"
Aug 05 18:19:42.429936 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:42.429834    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr" event={"ID":"e746138a-5050-4ec5-98db-304839bffc5c","Type":"ContainerStarted","Data":"584d20b978dbb731dfc664f078fc48a99d773d35cd932269ed3ebc69ee8d531f"}
Aug 05 18:19:42.446271 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:42.446232    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd" podStartSLOduration=1.446219004 podStartE2EDuration="1.446219004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:19:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:19:42.443934821 +0000 UTC m=+2789.796609418" watchObservedRunningTime="2026-08-05 18:19:42.446219004 +0000 UTC m=+2789.798893598"
Aug 05 18:19:42.463630 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:42.463593    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj" podStartSLOduration=2.032475612 podStartE2EDuration="2.463578814s" totalImagesPullingTime="431.103202ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:19:42.462957862 +0000 UTC m=+2789.815632458" watchObservedRunningTime="2026-08-05 18:19:42.463578814 +0000 UTC m=+2789.816253409"
Aug 05 18:19:42.491299 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:42.491266    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr" podStartSLOduration=1.491254741 podStartE2EDuration="1.491254741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:19:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:19:42.489162347 +0000 UTC m=+2789.841836941" watchObservedRunningTime="2026-08-05 18:19:42.491254741 +0000 UTC m=+2789.843929335"
Aug 05 18:19:48.441663 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:48.441636    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"
Aug 05 18:19:53.438481 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:53.438452    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr"
Aug 05 18:19:53.438921 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:19:53.438732    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd"
Aug 05 18:20:05.370624 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:05.370594    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:05.699212 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:05.699118    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:06.000228 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:06.000108    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:06.080002 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:06.079969    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:08.500960 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:08.500923    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:08.599787 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:08.599754    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:09.590735 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:09.590703    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"]
Aug 05 18:20:09.595284 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:09.595261    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"
Aug 05 18:20:09.597823 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:09.597799    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-6fb560-serving-cert\""
Aug 05 18:20:09.603366 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:09.603312    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"]
Aug 05 18:20:09.623460 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:09.623418    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:09.733956 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:09.733920    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tqmt\" (UniqueName: \"kubernetes.io/projected/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-kube-api-access-7tqmt\") pod \"maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq\" (UID: \"7ca7e691-e4ad-48a3-b3d3-e17d4858e002\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"
Aug 05 18:20:09.734090 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:09.734070    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-maas-api-tls\") pod \"maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq\" (UID: \"7ca7e691-e4ad-48a3-b3d3-e17d4858e002\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"
Aug 05 18:20:09.734147 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:09.734108    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-openshift-service-ca\") pod \"maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq\" (UID: \"7ca7e691-e4ad-48a3-b3d3-e17d4858e002\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"
Aug 05 18:20:09.835265 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:09.835240    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-maas-api-tls\") pod \"maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq\" (UID: \"7ca7e691-e4ad-48a3-b3d3-e17d4858e002\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"
Aug 05 18:20:09.835364 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:09.835278    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-openshift-service-ca\") pod \"maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq\" (UID: \"7ca7e691-e4ad-48a3-b3d3-e17d4858e002\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"
Aug 05 18:20:09.835364 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:09.835327    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tqmt\" (UniqueName: \"kubernetes.io/projected/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-kube-api-access-7tqmt\") pod \"maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq\" (UID: \"7ca7e691-e4ad-48a3-b3d3-e17d4858e002\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"
Aug 05 18:20:09.835479 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:20:09.835394    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-serving-cert: secret "maas-api-e2e-shared-b-6fb560-serving-cert" not found
Aug 05 18:20:09.835479 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:20:09.835469    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-maas-api-tls podName:7ca7e691-e4ad-48a3-b3d3-e17d4858e002 nodeName:}" failed. No retries permitted until 2026-08-05 18:20:10.335449652 +0000 UTC m=+2817.688124237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-maas-api-tls") pod "maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq" (UID: "7ca7e691-e4ad-48a3-b3d3-e17d4858e002") : secret "maas-api-e2e-shared-b-6fb560-serving-cert" not found
Aug 05 18:20:09.835966 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:09.835943    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-openshift-service-ca\") pod \"maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq\" (UID: \"7ca7e691-e4ad-48a3-b3d3-e17d4858e002\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"
Aug 05 18:20:09.845236 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:09.845170    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tqmt\" (UniqueName: \"kubernetes.io/projected/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-kube-api-access-7tqmt\") pod \"maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq\" (UID: \"7ca7e691-e4ad-48a3-b3d3-e17d4858e002\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"
Aug 05 18:20:09.980039 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:09.980000    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:10.131126 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:10.131047    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:10.235994 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:10.235944    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:10.341039 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:10.340980    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-maas-api-tls\") pod \"maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq\" (UID: \"7ca7e691-e4ad-48a3-b3d3-e17d4858e002\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"
Aug 05 18:20:10.342764 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:10.342740    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf"]
Aug 05 18:20:10.343768 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:10.343738    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-maas-api-tls\") pod \"maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq\" (UID: \"7ca7e691-e4ad-48a3-b3d3-e17d4858e002\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"
Aug 05 18:20:10.346651 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:10.346631    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf"
Aug 05 18:20:10.352271 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:10.352252    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-6fb560-dockercfg-h7q6j\""
Aug 05 18:20:10.352800 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:10.352784    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-6fb560\""
Aug 05 18:20:10.365931 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:10.365909    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf"]
Aug 05 18:20:10.441938 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:10.441828    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/133dc018-7d7b-463d-bbb9-d4e0cecfa306-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf\" (UID: \"133dc018-7d7b-463d-bbb9-d4e0cecfa306\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf"
Aug 05 18:20:10.441938 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:10.441922    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t29j\" (UniqueName: \"kubernetes.io/projected/133dc018-7d7b-463d-bbb9-d4e0cecfa306-kube-api-access-7t29j\") pod \"payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf\" (UID: \"133dc018-7d7b-463d-bbb9-d4e0cecfa306\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf"
Aug 05 18:20:10.507965 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:10.507925    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"
Aug 05 18:20:10.543746 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:10.543700    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/133dc018-7d7b-463d-bbb9-d4e0cecfa306-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf\" (UID: \"133dc018-7d7b-463d-bbb9-d4e0cecfa306\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf"
Aug 05 18:20:10.543911 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:10.543767    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t29j\" (UniqueName: \"kubernetes.io/projected/133dc018-7d7b-463d-bbb9-d4e0cecfa306-kube-api-access-7t29j\") pod \"payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf\" (UID: \"133dc018-7d7b-463d-bbb9-d4e0cecfa306\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf"
Aug 05 18:20:10.544397 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:10.544374    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/133dc018-7d7b-463d-bbb9-d4e0cecfa306-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf\" (UID: \"133dc018-7d7b-463d-bbb9-d4e0cecfa306\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf"
Aug 05 18:20:10.553308 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:10.553280    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t29j\" (UniqueName: \"kubernetes.io/projected/133dc018-7d7b-463d-bbb9-d4e0cecfa306-kube-api-access-7t29j\") pod \"payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf\" (UID: \"133dc018-7d7b-463d-bbb9-d4e0cecfa306\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf"
Aug 05 18:20:10.655054 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:10.655016    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"]
Aug 05 18:20:10.658945 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:20:10.658916    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ca7e691_e4ad_48a3_b3d3_e17d4858e002.slice/crio-84cd34ed46b84d41f1a652f6cf2f6c2d014cb806ee57e944fc85f9f73d448ca5 WatchSource:0}: Error finding container 84cd34ed46b84d41f1a652f6cf2f6c2d014cb806ee57e944fc85f9f73d448ca5: Status 404 returned error can't find the container with id 84cd34ed46b84d41f1a652f6cf2f6c2d014cb806ee57e944fc85f9f73d448ca5
Aug 05 18:20:10.664617 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:10.664580    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf"
Aug 05 18:20:10.809476 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:10.809448    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf"]
Aug 05 18:20:10.810792 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:20:10.810773    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod133dc018_7d7b_463d_bbb9_d4e0cecfa306.slice/crio-ab5411434da2766110fe18f86ba7623d28f5389fc1a772d525d648268b6aadeb WatchSource:0}: Error finding container ab5411434da2766110fe18f86ba7623d28f5389fc1a772d525d648268b6aadeb: Status 404 returned error can't find the container with id ab5411434da2766110fe18f86ba7623d28f5389fc1a772d525d648268b6aadeb
Aug 05 18:20:11.587390 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:11.587351    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq" event={"ID":"7ca7e691-e4ad-48a3-b3d3-e17d4858e002","Type":"ContainerStarted","Data":"5cd664d124081b4ec25b0089f08439cf68ef0a516d4a5f0046b0a938f9bc4c0c"}
Aug 05 18:20:11.587586 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:11.587411    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq" event={"ID":"7ca7e691-e4ad-48a3-b3d3-e17d4858e002","Type":"ContainerStarted","Data":"84cd34ed46b84d41f1a652f6cf2f6c2d014cb806ee57e944fc85f9f73d448ca5"}
Aug 05 18:20:11.587586 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:11.587455    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"
Aug 05 18:20:11.588933 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:11.588907    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf" event={"ID":"133dc018-7d7b-463d-bbb9-d4e0cecfa306","Type":"ContainerStarted","Data":"2a463a8da7c41d026dc43dbbab7fc0fb713d40c8739aa6aece18d2682d4a6821"}
Aug 05 18:20:11.589054 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:11.588946    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf" event={"ID":"133dc018-7d7b-463d-bbb9-d4e0cecfa306","Type":"ContainerStarted","Data":"ab5411434da2766110fe18f86ba7623d28f5389fc1a772d525d648268b6aadeb"}
Aug 05 18:20:11.589054 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:11.588994    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf"
Aug 05 18:20:11.611653 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:11.611613    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq" podStartSLOduration=2.271440166 podStartE2EDuration="2.611598799s" totalImagesPullingTime="340.158633ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:20:11.609303548 +0000 UTC m=+2818.961978142" watchObservedRunningTime="2026-08-05 18:20:11.611598799 +0000 UTC m=+2818.964273393"
Aug 05 18:20:11.627690 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:11.627646    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf" podStartSLOduration=1.627633151 podStartE2EDuration="1.627633151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:20:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:20:11.627082009 +0000 UTC m=+2818.979756604" watchObservedRunningTime="2026-08-05 18:20:11.627633151 +0000 UTC m=+2818.980307745"
Aug 05 18:20:17.600805 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:17.600771    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"
Aug 05 18:20:17.771762 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:17.771722    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:22.596984 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:22.596951    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf"
Aug 05 18:20:33.480474 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:33.480429    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:39.255135 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:39.255102    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:49.287384 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:49.287350    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:58.155578 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.155545    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"]
Aug 05 18:20:58.159300 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.159283    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.162786 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.162769    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6fb560\"/\"kube-root-ca.crt\""
Aug 05 18:20:58.164104 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.164085    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6fb560\"/\"auth-test-model-6fb560-kserve-self-signed-certs\""
Aug 05 18:20:58.164216 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.164130    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6fb560\"/\"openshift-service-ca.crt\""
Aug 05 18:20:58.164216 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.164156    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6fb560\"/\"default-dockercfg-zkz2j\""
Aug 05 18:20:58.171504 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.171480    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"]
Aug 05 18:20:58.182999 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.182968    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-home\") pod \"auth-test-model-6fb560-kserve-99486557f-gcg4v\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") " pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.183096 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.183020    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5818ba6-72ee-416a-ae65-42981666b19b-tls-certs\") pod \"auth-test-model-6fb560-kserve-99486557f-gcg4v\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") " pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.183141 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.183090    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlnlg\" (UniqueName: \"kubernetes.io/projected/b5818ba6-72ee-416a-ae65-42981666b19b-kube-api-access-dlnlg\") pod \"auth-test-model-6fb560-kserve-99486557f-gcg4v\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") " pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.183208 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.183167    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-model-cache\") pod \"auth-test-model-6fb560-kserve-99486557f-gcg4v\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") " pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.183296 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.183273    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-tmp-dir\") pod \"auth-test-model-6fb560-kserve-99486557f-gcg4v\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") " pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.183356 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.183310    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-dshm\") pod \"auth-test-model-6fb560-kserve-99486557f-gcg4v\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") " pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.188362 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.188335    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:58.284622 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.284595    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-home\") pod \"auth-test-model-6fb560-kserve-99486557f-gcg4v\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") " pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.284754 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.284627    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5818ba6-72ee-416a-ae65-42981666b19b-tls-certs\") pod \"auth-test-model-6fb560-kserve-99486557f-gcg4v\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") " pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.284754 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.284655    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlnlg\" (UniqueName: \"kubernetes.io/projected/b5818ba6-72ee-416a-ae65-42981666b19b-kube-api-access-dlnlg\") pod \"auth-test-model-6fb560-kserve-99486557f-gcg4v\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") " pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.284754 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.284703    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-model-cache\") pod \"auth-test-model-6fb560-kserve-99486557f-gcg4v\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") " pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.284754 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.284722    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-tmp-dir\") pod \"auth-test-model-6fb560-kserve-99486557f-gcg4v\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") " pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.284754 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.284738    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-dshm\") pod \"auth-test-model-6fb560-kserve-99486557f-gcg4v\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") " pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.285042 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.285010    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-home\") pod \"auth-test-model-6fb560-kserve-99486557f-gcg4v\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") " pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.285101 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.285066    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-model-cache\") pod \"auth-test-model-6fb560-kserve-99486557f-gcg4v\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") " pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.285137 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.285127    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-tmp-dir\") pod \"auth-test-model-6fb560-kserve-99486557f-gcg4v\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") " pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.287041 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.287024    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-dshm\") pod \"auth-test-model-6fb560-kserve-99486557f-gcg4v\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") " pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.287231 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.287216    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5818ba6-72ee-416a-ae65-42981666b19b-tls-certs\") pod \"auth-test-model-6fb560-kserve-99486557f-gcg4v\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") " pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.299569 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.299550    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlnlg\" (UniqueName: \"kubernetes.io/projected/b5818ba6-72ee-416a-ae65-42981666b19b-kube-api-access-dlnlg\") pod \"auth-test-model-6fb560-kserve-99486557f-gcg4v\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") " pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.469229 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.469154    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:20:58.607158 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.607132    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"]
Aug 05 18:20:58.609499 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:20:58.609472    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5818ba6_72ee_416a_ae65_42981666b19b.slice/crio-5b57b014e00150aa846f1cd93cf8a9032ddf3b7c5a3f3cca08ddc1fec1f31598 WatchSource:0}: Error finding container 5b57b014e00150aa846f1cd93cf8a9032ddf3b7c5a3f3cca08ddc1fec1f31598: Status 404 returned error can't find the container with id 5b57b014e00150aa846f1cd93cf8a9032ddf3b7c5a3f3cca08ddc1fec1f31598
Aug 05 18:20:58.834498 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.834470    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v" event={"ID":"b5818ba6-72ee-416a-ae65-42981666b19b","Type":"ContainerStarted","Data":"6e2f6d54eaa5126bc312bcce40f2d9ad48aecf56c7cc65edc20308a724c0892e"}
Aug 05 18:20:58.834654 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.834508    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v" event={"ID":"b5818ba6-72ee-416a-ae65-42981666b19b","Type":"ContainerStarted","Data":"5b57b014e00150aa846f1cd93cf8a9032ddf3b7c5a3f3cca08ddc1fec1f31598"}
Aug 05 18:20:58.854859 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:20:58.854815    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v" podStartSLOduration=0.854801178 podStartE2EDuration="854.801178ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:20:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:20:58.853112923 +0000 UTC m=+2866.205787518" watchObservedRunningTime="2026-08-05 18:20:58.854801178 +0000 UTC m=+2866.207475803"
Aug 05 18:21:08.470050 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:21:08.469979    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:21:08.470050 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:21:08.470032    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:21:08.482286 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:21:08.482262    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:21:08.902419 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:21:08.902393    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:23:07.693346 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:23:07.693304    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:23:13.742925 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:23:13.742880    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:24:28.345533 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:28.345492    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:24:28.445434 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:28.445401    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:24:34.625780 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:34.625746    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:24:35.519478 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:35.519435    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"]
Aug 05 18:24:35.534041 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:35.533999    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:24:50.540030 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:50.539979    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v" podUID="b5818ba6-72ee-416a-ae65-42981666b19b" containerName="main" containerID="cri-o://6e2f6d54eaa5126bc312bcce40f2d9ad48aecf56c7cc65edc20308a724c0892e" gracePeriod=45
Aug 05 18:24:50.795364 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:50.795308    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:24:50.978282 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:50.978251    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-dshm\") pod \"b5818ba6-72ee-416a-ae65-42981666b19b\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") "
Aug 05 18:24:50.978479 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:50.978334    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlnlg\" (UniqueName: \"kubernetes.io/projected/b5818ba6-72ee-416a-ae65-42981666b19b-kube-api-access-dlnlg\") pod \"b5818ba6-72ee-416a-ae65-42981666b19b\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") "
Aug 05 18:24:50.978479 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:50.978369    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-tmp-dir\") pod \"b5818ba6-72ee-416a-ae65-42981666b19b\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") "
Aug 05 18:24:50.978479 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:50.978416    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-home\") pod \"b5818ba6-72ee-416a-ae65-42981666b19b\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") "
Aug 05 18:24:50.978479 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:50.978438    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5818ba6-72ee-416a-ae65-42981666b19b-tls-certs\") pod \"b5818ba6-72ee-416a-ae65-42981666b19b\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") "
Aug 05 18:24:50.978716 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:50.978483    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-model-cache\") pod \"b5818ba6-72ee-416a-ae65-42981666b19b\" (UID: \"b5818ba6-72ee-416a-ae65-42981666b19b\") "
Aug 05 18:24:50.978716 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:50.978626    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-home" (OuterVolumeSpecName: "home") pod "b5818ba6-72ee-416a-ae65-42981666b19b" (UID: "b5818ba6-72ee-416a-ae65-42981666b19b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:24:50.978716 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:50.978656    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b5818ba6-72ee-416a-ae65-42981666b19b" (UID: "b5818ba6-72ee-416a-ae65-42981666b19b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:24:50.978880 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:50.978830    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-model-cache" (OuterVolumeSpecName: "model-cache") pod "b5818ba6-72ee-416a-ae65-42981666b19b" (UID: "b5818ba6-72ee-416a-ae65-42981666b19b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:24:50.978939 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:50.978894    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-tmp-dir\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:24:50.978939 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:50.978914    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-home\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:24:50.980724 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:50.980697    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-dshm" (OuterVolumeSpecName: "dshm") pod "b5818ba6-72ee-416a-ae65-42981666b19b" (UID: "b5818ba6-72ee-416a-ae65-42981666b19b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:24:50.980724 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:50.980700    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5818ba6-72ee-416a-ae65-42981666b19b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b5818ba6-72ee-416a-ae65-42981666b19b" (UID: "b5818ba6-72ee-416a-ae65-42981666b19b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:24:50.980865 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:50.980740    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5818ba6-72ee-416a-ae65-42981666b19b-kube-api-access-dlnlg" (OuterVolumeSpecName: "kube-api-access-dlnlg") pod "b5818ba6-72ee-416a-ae65-42981666b19b" (UID: "b5818ba6-72ee-416a-ae65-42981666b19b"). InnerVolumeSpecName "kube-api-access-dlnlg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:24:51.079476 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:51.079450    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5818ba6-72ee-416a-ae65-42981666b19b-tls-certs\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:24:51.079476 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:51.079474    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-model-cache\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:24:51.079476 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:51.079482    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5818ba6-72ee-416a-ae65-42981666b19b-dshm\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:24:51.079728 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:51.079490    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlnlg\" (UniqueName: \"kubernetes.io/projected/b5818ba6-72ee-416a-ae65-42981666b19b-kube-api-access-dlnlg\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:24:51.079813 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:51.079794    2569 generic.go:356] "Generic (PLEG): container finished" podID="b5818ba6-72ee-416a-ae65-42981666b19b" containerID="6e2f6d54eaa5126bc312bcce40f2d9ad48aecf56c7cc65edc20308a724c0892e" exitCode=0
Aug 05 18:24:51.079859 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:51.079844    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"
Aug 05 18:24:51.079895 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:51.079867    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v" event={"ID":"b5818ba6-72ee-416a-ae65-42981666b19b","Type":"ContainerDied","Data":"6e2f6d54eaa5126bc312bcce40f2d9ad48aecf56c7cc65edc20308a724c0892e"}
Aug 05 18:24:51.079928 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:51.079901    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v" event={"ID":"b5818ba6-72ee-416a-ae65-42981666b19b","Type":"ContainerDied","Data":"5b57b014e00150aa846f1cd93cf8a9032ddf3b7c5a3f3cca08ddc1fec1f31598"}
Aug 05 18:24:51.079928 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:51.079918    2569 scope.go:117] "RemoveContainer" containerID="6e2f6d54eaa5126bc312bcce40f2d9ad48aecf56c7cc65edc20308a724c0892e"
Aug 05 18:24:51.089230 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:51.089211    2569 scope.go:117] "RemoveContainer" containerID="6e2f6d54eaa5126bc312bcce40f2d9ad48aecf56c7cc65edc20308a724c0892e"
Aug 05 18:24:51.089492 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:24:51.089474    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e2f6d54eaa5126bc312bcce40f2d9ad48aecf56c7cc65edc20308a724c0892e\": container with ID starting with 6e2f6d54eaa5126bc312bcce40f2d9ad48aecf56c7cc65edc20308a724c0892e not found: ID does not exist" containerID="6e2f6d54eaa5126bc312bcce40f2d9ad48aecf56c7cc65edc20308a724c0892e"
Aug 05 18:24:51.089537 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:51.089501    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e2f6d54eaa5126bc312bcce40f2d9ad48aecf56c7cc65edc20308a724c0892e"} err="failed to get container status \"6e2f6d54eaa5126bc312bcce40f2d9ad48aecf56c7cc65edc20308a724c0892e\": rpc error: code = NotFound desc = could not find container \"6e2f6d54eaa5126bc312bcce40f2d9ad48aecf56c7cc65edc20308a724c0892e\": container with ID starting with 6e2f6d54eaa5126bc312bcce40f2d9ad48aecf56c7cc65edc20308a724c0892e not found: ID does not exist"
Aug 05 18:24:51.101657 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:51.101637    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"]
Aug 05 18:24:51.105740 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:51.105722    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-6fb560/auth-test-model-6fb560-kserve-99486557f-gcg4v"]
Aug 05 18:24:51.150720 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:51.150695    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5818ba6-72ee-416a-ae65-42981666b19b" path="/var/lib/kubelet/pods/b5818ba6-72ee-416a-ae65-42981666b19b/volumes"
Aug 05 18:24:52.566124 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:52.566090    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:24:57.050248 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:24:57.050210    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:25:23.642883 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.642847    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"]
Aug 05 18:25:23.643562 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.643537    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5818ba6-72ee-416a-ae65-42981666b19b" containerName="main"
Aug 05 18:25:23.643562 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.643560    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5818ba6-72ee-416a-ae65-42981666b19b" containerName="main"
Aug 05 18:25:23.643767 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.643684    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5818ba6-72ee-416a-ae65-42981666b19b" containerName="main"
Aug 05 18:25:23.646958 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.646941    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:23.649701 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.649680    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6fb560\"/\"openshift-service-ca.crt\""
Aug 05 18:25:23.649818 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.649702    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6fb560\"/\"default-dockercfg-zkz2j\""
Aug 05 18:25:23.650868 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.650849    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6fb560\"/\"sub-test-model-6fb560-kserve-self-signed-certs\""
Aug 05 18:25:23.650938 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.650852    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6fb560\"/\"kube-root-ca.crt\""
Aug 05 18:25:23.656041 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.656016    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"]
Aug 05 18:25:23.683994 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.683954    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:25:23.781961 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.781921    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x4lv\" (UniqueName: \"kubernetes.io/projected/2e65cede-ab2f-43e5-8197-6b7f8c871356-kube-api-access-5x4lv\") pod \"sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") " pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:23.782155 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.781965    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e65cede-ab2f-43e5-8197-6b7f8c871356-tls-certs\") pod \"sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") " pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:23.782155 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.782041    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-model-cache\") pod \"sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") " pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:23.782155 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.782075    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-tmp-dir\") pod \"sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") " pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:23.782155 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.782105    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-dshm\") pod \"sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") " pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:23.782347 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.782207    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-home\") pod \"sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") " pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:23.883412 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.883369    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-home\") pod \"sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") " pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:23.883620 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.883435    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x4lv\" (UniqueName: \"kubernetes.io/projected/2e65cede-ab2f-43e5-8197-6b7f8c871356-kube-api-access-5x4lv\") pod \"sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") " pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:23.883620 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.883461    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e65cede-ab2f-43e5-8197-6b7f8c871356-tls-certs\") pod \"sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") " pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:23.883620 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.883515    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-model-cache\") pod \"sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") " pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:23.883620 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.883544    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-tmp-dir\") pod \"sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") " pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:23.883620 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.883587    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-dshm\") pod \"sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") " pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:23.883911 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.883794    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-home\") pod \"sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") " pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:23.883972 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.883921    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-model-cache\") pod \"sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") " pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:23.883972 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.883927    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-tmp-dir\") pod \"sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") " pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:23.886038 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.886007    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-dshm\") pod \"sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") " pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:23.886257 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.886239    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e65cede-ab2f-43e5-8197-6b7f8c871356-tls-certs\") pod \"sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") " pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:23.892680 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.892656    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x4lv\" (UniqueName: \"kubernetes.io/projected/2e65cede-ab2f-43e5-8197-6b7f8c871356-kube-api-access-5x4lv\") pod \"sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") " pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:23.957851 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:23.957754    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:24.124775 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:24.124744    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"]
Aug 05 18:25:24.127169 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:25:24.127136    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e65cede_ab2f_43e5_8197_6b7f8c871356.slice/crio-76411d41d3b1dc6f08482092bc713f5648ee5160158e7820f5e63aae7f665f33 WatchSource:0}: Error finding container 76411d41d3b1dc6f08482092bc713f5648ee5160158e7820f5e63aae7f665f33: Status 404 returned error can't find the container with id 76411d41d3b1dc6f08482092bc713f5648ee5160158e7820f5e63aae7f665f33
Aug 05 18:25:24.267304 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:24.267206    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc" event={"ID":"2e65cede-ab2f-43e5-8197-6b7f8c871356","Type":"ContainerStarted","Data":"078c0b2ab6b017730217f2d8b463256fc27f897dd7038b782563a3457f9949fc"}
Aug 05 18:25:24.267304 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:24.267258    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc" event={"ID":"2e65cede-ab2f-43e5-8197-6b7f8c871356","Type":"ContainerStarted","Data":"76411d41d3b1dc6f08482092bc713f5648ee5160158e7820f5e63aae7f665f33"}
Aug 05 18:25:24.298760 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:24.298711    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc" podStartSLOduration=1.298693999 podStartE2EDuration="1.298693999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:25:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:25:24.29687253 +0000 UTC m=+3131.649547125" watchObservedRunningTime="2026-08-05 18:25:24.298693999 +0000 UTC m=+3131.651368592"
Aug 05 18:25:33.958973 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:33.958895    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:33.958973 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:33.958952    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:33.971853 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:33.971827    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:25:34.332677 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:25:34.332651    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:26:38.333846 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:38.333811    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:26:39.144755 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:39.144720    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"]
Aug 05 18:26:39.169978 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:39.169941    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:26:47.434901 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:47.434864    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:26:54.166078 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.166038    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc" podUID="2e65cede-ab2f-43e5-8197-6b7f8c871356" containerName="main" containerID="cri-o://078c0b2ab6b017730217f2d8b463256fc27f897dd7038b782563a3457f9949fc" gracePeriod=45
Aug 05 18:26:54.322660 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.322610    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc" podUID="2e65cede-ab2f-43e5-8197-6b7f8c871356" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": dial tcp 10.134.0.81:8000: connect: connection refused"
Aug 05 18:26:54.419008 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.418953    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:26:54.524704 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.524672    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-model-cache\") pod \"2e65cede-ab2f-43e5-8197-6b7f8c871356\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") "
Aug 05 18:26:54.524850 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.524731    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-tmp-dir\") pod \"2e65cede-ab2f-43e5-8197-6b7f8c871356\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") "
Aug 05 18:26:54.524850 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.524770    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5x4lv\" (UniqueName: \"kubernetes.io/projected/2e65cede-ab2f-43e5-8197-6b7f8c871356-kube-api-access-5x4lv\") pod \"2e65cede-ab2f-43e5-8197-6b7f8c871356\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") "
Aug 05 18:26:54.524850 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.524810    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e65cede-ab2f-43e5-8197-6b7f8c871356-tls-certs\") pod \"2e65cede-ab2f-43e5-8197-6b7f8c871356\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") "
Aug 05 18:26:54.524850 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.524843    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-dshm\") pod \"2e65cede-ab2f-43e5-8197-6b7f8c871356\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") "
Aug 05 18:26:54.525018 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.524861    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-home\") pod \"2e65cede-ab2f-43e5-8197-6b7f8c871356\" (UID: \"2e65cede-ab2f-43e5-8197-6b7f8c871356\") "
Aug 05 18:26:54.525018 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.524929    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-model-cache" (OuterVolumeSpecName: "model-cache") pod "2e65cede-ab2f-43e5-8197-6b7f8c871356" (UID: "2e65cede-ab2f-43e5-8197-6b7f8c871356"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:26:54.525127 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.525029    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2e65cede-ab2f-43e5-8197-6b7f8c871356" (UID: "2e65cede-ab2f-43e5-8197-6b7f8c871356"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:26:54.525291 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.525264    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-home" (OuterVolumeSpecName: "home") pod "2e65cede-ab2f-43e5-8197-6b7f8c871356" (UID: "2e65cede-ab2f-43e5-8197-6b7f8c871356"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:26:54.525392 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.525275    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-model-cache\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:26:54.525392 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.525328    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-tmp-dir\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:26:54.527035 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.527014    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-dshm" (OuterVolumeSpecName: "dshm") pod "2e65cede-ab2f-43e5-8197-6b7f8c871356" (UID: "2e65cede-ab2f-43e5-8197-6b7f8c871356"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:26:54.527125 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.527048    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e65cede-ab2f-43e5-8197-6b7f8c871356-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2e65cede-ab2f-43e5-8197-6b7f8c871356" (UID: "2e65cede-ab2f-43e5-8197-6b7f8c871356"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:26:54.527201 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.527148    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e65cede-ab2f-43e5-8197-6b7f8c871356-kube-api-access-5x4lv" (OuterVolumeSpecName: "kube-api-access-5x4lv") pod "2e65cede-ab2f-43e5-8197-6b7f8c871356" (UID: "2e65cede-ab2f-43e5-8197-6b7f8c871356"). InnerVolumeSpecName "kube-api-access-5x4lv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:26:54.626405 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.626381    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5x4lv\" (UniqueName: \"kubernetes.io/projected/2e65cede-ab2f-43e5-8197-6b7f8c871356-kube-api-access-5x4lv\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:26:54.626405 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.626403    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e65cede-ab2f-43e5-8197-6b7f8c871356-tls-certs\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:26:54.626537 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.626413    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-dshm\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:26:54.626537 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.626421    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e65cede-ab2f-43e5-8197-6b7f8c871356-home\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:26:54.741405 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.741341    2569 generic.go:356] "Generic (PLEG): container finished" podID="2e65cede-ab2f-43e5-8197-6b7f8c871356" containerID="078c0b2ab6b017730217f2d8b463256fc27f897dd7038b782563a3457f9949fc" exitCode=0
Aug 05 18:26:54.741405 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.741378    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc" event={"ID":"2e65cede-ab2f-43e5-8197-6b7f8c871356","Type":"ContainerDied","Data":"078c0b2ab6b017730217f2d8b463256fc27f897dd7038b782563a3457f9949fc"}
Aug 05 18:26:54.741405 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.741395    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"
Aug 05 18:26:54.741616 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.741421    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc" event={"ID":"2e65cede-ab2f-43e5-8197-6b7f8c871356","Type":"ContainerDied","Data":"76411d41d3b1dc6f08482092bc713f5648ee5160158e7820f5e63aae7f665f33"}
Aug 05 18:26:54.741616 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.741438    2569 scope.go:117] "RemoveContainer" containerID="078c0b2ab6b017730217f2d8b463256fc27f897dd7038b782563a3457f9949fc"
Aug 05 18:26:54.751654 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.751639    2569 scope.go:117] "RemoveContainer" containerID="078c0b2ab6b017730217f2d8b463256fc27f897dd7038b782563a3457f9949fc"
Aug 05 18:26:54.751912 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:26:54.751890    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"078c0b2ab6b017730217f2d8b463256fc27f897dd7038b782563a3457f9949fc\": container with ID starting with 078c0b2ab6b017730217f2d8b463256fc27f897dd7038b782563a3457f9949fc not found: ID does not exist" containerID="078c0b2ab6b017730217f2d8b463256fc27f897dd7038b782563a3457f9949fc"
Aug 05 18:26:54.751971 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.751926    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"078c0b2ab6b017730217f2d8b463256fc27f897dd7038b782563a3457f9949fc"} err="failed to get container status \"078c0b2ab6b017730217f2d8b463256fc27f897dd7038b782563a3457f9949fc\": rpc error: code = NotFound desc = could not find container \"078c0b2ab6b017730217f2d8b463256fc27f897dd7038b782563a3457f9949fc\": container with ID starting with 078c0b2ab6b017730217f2d8b463256fc27f897dd7038b782563a3457f9949fc not found: ID does not exist"
Aug 05 18:26:54.763068 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.763040    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"]
Aug 05 18:26:54.765163 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:54.765144    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-6fb560/sub-test-model-6fb560-kserve-bdc5d4b5-rc9bc"]
Aug 05 18:26:55.152003 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:26:55.151968    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e65cede-ab2f-43e5-8197-6b7f8c871356" path="/var/lib/kubelet/pods/2e65cede-ab2f-43e5-8197-6b7f8c871356/volumes"
Aug 05 18:27:07.424124 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.424039    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"]
Aug 05 18:27:07.424673 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.424654    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e65cede-ab2f-43e5-8197-6b7f8c871356" containerName="main"
Aug 05 18:27:07.424764 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.424676    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e65cede-ab2f-43e5-8197-6b7f8c871356" containerName="main"
Aug 05 18:27:07.424822 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.424785    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e65cede-ab2f-43e5-8197-6b7f8c871356" containerName="main"
Aug 05 18:27:07.429481 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.429456    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.443129 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.443110    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6fb560\"/\"rate-test-model-6fb560-kserve-self-signed-certs\""
Aug 05 18:27:07.447372 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.447352    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6fb560\"/\"kube-root-ca.crt\""
Aug 05 18:27:07.447372 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.447375    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6fb560\"/\"default-dockercfg-zkz2j\""
Aug 05 18:27:07.447871 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.447854    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6fb560\"/\"openshift-service-ca.crt\""
Aug 05 18:27:07.449122 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.449099    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"]
Aug 05 18:27:07.508049 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.508029    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:27:07.545250 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.545219    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6814ce0c-94c5-4010-96aa-69127a559d59-tls-certs\") pod \"rate-test-model-6fb560-kserve-67fff4dc6c-njncr\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") " pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.545375 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.545270    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-model-cache\") pod \"rate-test-model-6fb560-kserve-67fff4dc6c-njncr\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") " pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.545510 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.545494    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-home\") pod \"rate-test-model-6fb560-kserve-67fff4dc6c-njncr\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") " pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.545595 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.545561    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7gkk\" (UniqueName: \"kubernetes.io/projected/6814ce0c-94c5-4010-96aa-69127a559d59-kube-api-access-c7gkk\") pod \"rate-test-model-6fb560-kserve-67fff4dc6c-njncr\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") " pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.545655 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.545608    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-tmp-dir\") pod \"rate-test-model-6fb560-kserve-67fff4dc6c-njncr\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") " pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.545715 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.545674    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-dshm\") pod \"rate-test-model-6fb560-kserve-67fff4dc6c-njncr\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") " pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.646963 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.646933    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-dshm\") pod \"rate-test-model-6fb560-kserve-67fff4dc6c-njncr\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") " pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.647089 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.647005    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6814ce0c-94c5-4010-96aa-69127a559d59-tls-certs\") pod \"rate-test-model-6fb560-kserve-67fff4dc6c-njncr\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") " pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.647089 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.647027    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-model-cache\") pod \"rate-test-model-6fb560-kserve-67fff4dc6c-njncr\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") " pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.647089 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.647051    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-home\") pod \"rate-test-model-6fb560-kserve-67fff4dc6c-njncr\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") " pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.647246 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.647103    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7gkk\" (UniqueName: \"kubernetes.io/projected/6814ce0c-94c5-4010-96aa-69127a559d59-kube-api-access-c7gkk\") pod \"rate-test-model-6fb560-kserve-67fff4dc6c-njncr\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") " pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.647246 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.647150    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-tmp-dir\") pod \"rate-test-model-6fb560-kserve-67fff4dc6c-njncr\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") " pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.647506 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.647473    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-model-cache\") pod \"rate-test-model-6fb560-kserve-67fff4dc6c-njncr\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") " pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.647632 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.647521    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-home\") pod \"rate-test-model-6fb560-kserve-67fff4dc6c-njncr\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") " pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.647632 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.647551    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-tmp-dir\") pod \"rate-test-model-6fb560-kserve-67fff4dc6c-njncr\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") " pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.649274 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.649252    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-dshm\") pod \"rate-test-model-6fb560-kserve-67fff4dc6c-njncr\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") " pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.649603 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.649585    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6814ce0c-94c5-4010-96aa-69127a559d59-tls-certs\") pod \"rate-test-model-6fb560-kserve-67fff4dc6c-njncr\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") " pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.664164 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.664143    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7gkk\" (UniqueName: \"kubernetes.io/projected/6814ce0c-94c5-4010-96aa-69127a559d59-kube-api-access-c7gkk\") pod \"rate-test-model-6fb560-kserve-67fff4dc6c-njncr\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") " pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.739170 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.739114    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:07.881351 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:07.881325    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"]
Aug 05 18:27:07.885346 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:27:07.884934    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6814ce0c_94c5_4010_96aa_69127a559d59.slice/crio-748225abd6ec8862e75894b3e2cc56ddda33d599f31247a302c3c62d431b34f5 WatchSource:0}: Error finding container 748225abd6ec8862e75894b3e2cc56ddda33d599f31247a302c3c62d431b34f5: Status 404 returned error can't find the container with id 748225abd6ec8862e75894b3e2cc56ddda33d599f31247a302c3c62d431b34f5
Aug 05 18:27:08.818169 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:08.818129    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr" event={"ID":"6814ce0c-94c5-4010-96aa-69127a559d59","Type":"ContainerStarted","Data":"ca664cc30b2d96b2fd3f013ed8860c69c9e9efdf1a1d435985ddad59465de16b"}
Aug 05 18:27:08.818596 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:08.818193    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr" event={"ID":"6814ce0c-94c5-4010-96aa-69127a559d59","Type":"ContainerStarted","Data":"748225abd6ec8862e75894b3e2cc56ddda33d599f31247a302c3c62d431b34f5"}
Aug 05 18:27:08.837973 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:08.837926    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr" podStartSLOduration=1.837908818 podStartE2EDuration="1.837908818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:27:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:27:08.837405601 +0000 UTC m=+3236.190080196" watchObservedRunningTime="2026-08-05 18:27:08.837908818 +0000 UTC m=+3236.190583413"
Aug 05 18:27:17.740171 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:17.740134    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:17.740624 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:17.740219    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:17.752864 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:17.752843    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:27:17.877956 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:27:17.877935    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:28:26.533938 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:26.533907    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:28:26.650087 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:26.650052    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"]
Aug 05 18:28:28.284048 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:28.284017    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:28:40.865743 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:40.865657    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:28:41.669120 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:41.669079    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr" podUID="6814ce0c-94c5-4010-96aa-69127a559d59" containerName="main" containerID="cri-o://ca664cc30b2d96b2fd3f013ed8860c69c9e9efdf1a1d435985ddad59465de16b" gracePeriod=45
Aug 05 18:28:41.912267 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:41.912243    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:28:42.072291 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.072267    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-home\") pod \"6814ce0c-94c5-4010-96aa-69127a559d59\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") "
Aug 05 18:28:42.072450 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.072342    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-model-cache\") pod \"6814ce0c-94c5-4010-96aa-69127a559d59\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") "
Aug 05 18:28:42.072450 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.072402    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6814ce0c-94c5-4010-96aa-69127a559d59-tls-certs\") pod \"6814ce0c-94c5-4010-96aa-69127a559d59\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") "
Aug 05 18:28:42.072450 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.072429    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-tmp-dir\") pod \"6814ce0c-94c5-4010-96aa-69127a559d59\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") "
Aug 05 18:28:42.072561 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.072448    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7gkk\" (UniqueName: \"kubernetes.io/projected/6814ce0c-94c5-4010-96aa-69127a559d59-kube-api-access-c7gkk\") pod \"6814ce0c-94c5-4010-96aa-69127a559d59\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") "
Aug 05 18:28:42.072561 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.072467    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-dshm\") pod \"6814ce0c-94c5-4010-96aa-69127a559d59\" (UID: \"6814ce0c-94c5-4010-96aa-69127a559d59\") "
Aug 05 18:28:42.072561 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.072524    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-home" (OuterVolumeSpecName: "home") pod "6814ce0c-94c5-4010-96aa-69127a559d59" (UID: "6814ce0c-94c5-4010-96aa-69127a559d59"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:28:42.072673 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.072612    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-model-cache" (OuterVolumeSpecName: "model-cache") pod "6814ce0c-94c5-4010-96aa-69127a559d59" (UID: "6814ce0c-94c5-4010-96aa-69127a559d59"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:28:42.072722 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.072703    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6814ce0c-94c5-4010-96aa-69127a559d59" (UID: "6814ce0c-94c5-4010-96aa-69127a559d59"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:28:42.072869 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.072850    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-model-cache\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:28:42.072869 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.072868    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-tmp-dir\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:28:42.072869 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.072876    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-home\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:28:42.074777 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.074750    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6814ce0c-94c5-4010-96aa-69127a559d59-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6814ce0c-94c5-4010-96aa-69127a559d59" (UID: "6814ce0c-94c5-4010-96aa-69127a559d59"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:28:42.074903 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.074849    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-dshm" (OuterVolumeSpecName: "dshm") pod "6814ce0c-94c5-4010-96aa-69127a559d59" (UID: "6814ce0c-94c5-4010-96aa-69127a559d59"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:28:42.074903 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.074852    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6814ce0c-94c5-4010-96aa-69127a559d59-kube-api-access-c7gkk" (OuterVolumeSpecName: "kube-api-access-c7gkk") pod "6814ce0c-94c5-4010-96aa-69127a559d59" (UID: "6814ce0c-94c5-4010-96aa-69127a559d59"). InnerVolumeSpecName "kube-api-access-c7gkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:28:42.173965 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.173940    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6814ce0c-94c5-4010-96aa-69127a559d59-tls-certs\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:28:42.173965 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.173963    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7gkk\" (UniqueName: \"kubernetes.io/projected/6814ce0c-94c5-4010-96aa-69127a559d59-kube-api-access-c7gkk\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:28:42.173965 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.173972    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6814ce0c-94c5-4010-96aa-69127a559d59-dshm\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:28:42.303421 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.303384    2569 generic.go:356] "Generic (PLEG): container finished" podID="6814ce0c-94c5-4010-96aa-69127a559d59" containerID="ca664cc30b2d96b2fd3f013ed8860c69c9e9efdf1a1d435985ddad59465de16b" exitCode=0
Aug 05 18:28:42.303601 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.303432    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"
Aug 05 18:28:42.303601 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.303473    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr" event={"ID":"6814ce0c-94c5-4010-96aa-69127a559d59","Type":"ContainerDied","Data":"ca664cc30b2d96b2fd3f013ed8860c69c9e9efdf1a1d435985ddad59465de16b"}
Aug 05 18:28:42.303601 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.303532    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr" event={"ID":"6814ce0c-94c5-4010-96aa-69127a559d59","Type":"ContainerDied","Data":"748225abd6ec8862e75894b3e2cc56ddda33d599f31247a302c3c62d431b34f5"}
Aug 05 18:28:42.303601 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.303558    2569 scope.go:117] "RemoveContainer" containerID="ca664cc30b2d96b2fd3f013ed8860c69c9e9efdf1a1d435985ddad59465de16b"
Aug 05 18:28:42.313874 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.313857    2569 scope.go:117] "RemoveContainer" containerID="ca664cc30b2d96b2fd3f013ed8860c69c9e9efdf1a1d435985ddad59465de16b"
Aug 05 18:28:42.314096 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:28:42.314079    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca664cc30b2d96b2fd3f013ed8860c69c9e9efdf1a1d435985ddad59465de16b\": container with ID starting with ca664cc30b2d96b2fd3f013ed8860c69c9e9efdf1a1d435985ddad59465de16b not found: ID does not exist" containerID="ca664cc30b2d96b2fd3f013ed8860c69c9e9efdf1a1d435985ddad59465de16b"
Aug 05 18:28:42.314142 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.314105    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca664cc30b2d96b2fd3f013ed8860c69c9e9efdf1a1d435985ddad59465de16b"} err="failed to get container status \"ca664cc30b2d96b2fd3f013ed8860c69c9e9efdf1a1d435985ddad59465de16b\": rpc error: code = NotFound desc = could not find container \"ca664cc30b2d96b2fd3f013ed8860c69c9e9efdf1a1d435985ddad59465de16b\": container with ID starting with ca664cc30b2d96b2fd3f013ed8860c69c9e9efdf1a1d435985ddad59465de16b not found: ID does not exist"
Aug 05 18:28:42.325582 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.325533    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"]
Aug 05 18:28:42.332012 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:42.331990    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-6fb560/rate-test-model-6fb560-kserve-67fff4dc6c-njncr"]
Aug 05 18:28:43.150869 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:43.150838    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6814ce0c-94c5-4010-96aa-69127a559d59" path="/var/lib/kubelet/pods/6814ce0c-94c5-4010-96aa-69127a559d59/volumes"
Aug 05 18:28:45.756611 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:45.756570    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:28:46.067560 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:46.067534    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:28:46.318347 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:46.318277    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:28:48.803981 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:48.803952    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:28:48.896119 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:48.896089    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:28:49.944857 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:49.944819    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:28:50.218655 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:50.218573    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:28:50.530229 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:50.530131    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:28:50.630382 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:50.630351    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:28:50.670336 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:50.670308    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc"]
Aug 05 18:28:50.670758 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:50.670745    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6814ce0c-94c5-4010-96aa-69127a559d59" containerName="main"
Aug 05 18:28:50.670816 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:50.670762    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6814ce0c-94c5-4010-96aa-69127a559d59" containerName="main"
Aug 05 18:28:50.670898 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:50.670885    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6814ce0c-94c5-4010-96aa-69127a559d59" containerName="main"
Aug 05 18:28:50.675166 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:50.675150    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc"
Aug 05 18:28:50.678234 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:50.678215    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-b5b093\""
Aug 05 18:28:50.678314 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:50.678257    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-b5b093-dockercfg-lhrbz\""
Aug 05 18:28:50.688165 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:50.688144    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc"]
Aug 05 18:28:50.744083 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:50.744054    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cf7f\" (UniqueName: \"kubernetes.io/projected/dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db-kube-api-access-9cf7f\") pod \"payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc\" (UID: \"dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc"
Aug 05 18:28:50.744370 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:50.744345    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc\" (UID: \"dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc"
Aug 05 18:28:50.845822 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:50.845793    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc\" (UID: \"dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc"
Aug 05 18:28:50.845993 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:50.845868    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cf7f\" (UniqueName: \"kubernetes.io/projected/dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db-kube-api-access-9cf7f\") pod \"payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc\" (UID: \"dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc"
Aug 05 18:28:50.846574 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:50.846551    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc\" (UID: \"dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc"
Aug 05 18:28:50.855656 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:50.855629    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cf7f\" (UniqueName: \"kubernetes.io/projected/dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db-kube-api-access-9cf7f\") pod \"payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc\" (UID: \"dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc"
Aug 05 18:28:50.986315 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:50.986282    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc"
Aug 05 18:28:51.130245 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:51.130217    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc"]
Aug 05 18:28:51.131950 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:28:51.131925    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc805f0e_dda3_44b6_bc7b_9a7b6e73a7db.slice/crio-898430b817012499db184faeb4f2aca8fb7d53d692061843c0987f33c016a8a4 WatchSource:0}: Error finding container 898430b817012499db184faeb4f2aca8fb7d53d692061843c0987f33c016a8a4: Status 404 returned error can't find the container with id 898430b817012499db184faeb4f2aca8fb7d53d692061843c0987f33c016a8a4
Aug 05 18:28:51.363819 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:51.363781    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc" event={"ID":"dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db","Type":"ContainerStarted","Data":"e9bd87381b6b43f5f0696954fba22376f24f2f40b3e46ff8fcfa577aa8e1ace1"}
Aug 05 18:28:51.364008 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:51.363845    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc" event={"ID":"dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db","Type":"ContainerStarted","Data":"898430b817012499db184faeb4f2aca8fb7d53d692061843c0987f33c016a8a4"}
Aug 05 18:28:51.364008 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:51.363877    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc"
Aug 05 18:28:51.388434 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:28:51.388334    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc" podStartSLOduration=1.3883095349999999 podStartE2EDuration="1.388309535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:28:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:28:51.386478508 +0000 UTC m=+3338.739153115" watchObservedRunningTime="2026-08-05 18:28:51.388309535 +0000 UTC m=+3338.740984139"
Aug 05 18:29:02.371497 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:02.371466    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc"
Aug 05 18:29:13.732878 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:13.732845    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:29:14.377583 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:14.377552    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:29:14.668881 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:14.668800    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:29:14.806841 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:14.806795    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:29:17.939054 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:17.939015    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:29:18.042327 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:18.042294    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:29:18.990902 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:18.990869    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"]
Aug 05 18:29:18.995961 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:18.995936    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"
Aug 05 18:29:19.002689 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:19.002667    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-cea6f9-serving-cert\""
Aug 05 18:29:19.006149 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:19.006126    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"]
Aug 05 18:29:19.034832 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:19.034806    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:29:19.101751 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:19.101725    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10015b7a-34e1-4c8a-be41-f9c63e942cfe-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld\" (UID: \"10015b7a-34e1-4c8a-be41-f9c63e942cfe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"
Aug 05 18:29:19.101916 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:19.101784    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59xtg\" (UniqueName: \"kubernetes.io/projected/10015b7a-34e1-4c8a-be41-f9c63e942cfe-kube-api-access-59xtg\") pod \"maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld\" (UID: \"10015b7a-34e1-4c8a-be41-f9c63e942cfe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"
Aug 05 18:29:19.101916 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:19.101807    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10015b7a-34e1-4c8a-be41-f9c63e942cfe-maas-api-tls\") pod \"maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld\" (UID: \"10015b7a-34e1-4c8a-be41-f9c63e942cfe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"
Aug 05 18:29:19.202741 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:19.202713    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10015b7a-34e1-4c8a-be41-f9c63e942cfe-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld\" (UID: \"10015b7a-34e1-4c8a-be41-f9c63e942cfe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"
Aug 05 18:29:19.202931 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:19.202771    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59xtg\" (UniqueName: \"kubernetes.io/projected/10015b7a-34e1-4c8a-be41-f9c63e942cfe-kube-api-access-59xtg\") pod \"maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld\" (UID: \"10015b7a-34e1-4c8a-be41-f9c63e942cfe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"
Aug 05 18:29:19.202931 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:19.202794    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10015b7a-34e1-4c8a-be41-f9c63e942cfe-maas-api-tls\") pod \"maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld\" (UID: \"10015b7a-34e1-4c8a-be41-f9c63e942cfe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"
Aug 05 18:29:19.203369 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:19.203346    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10015b7a-34e1-4c8a-be41-f9c63e942cfe-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld\" (UID: \"10015b7a-34e1-4c8a-be41-f9c63e942cfe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"
Aug 05 18:29:19.205377 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:19.205359    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10015b7a-34e1-4c8a-be41-f9c63e942cfe-maas-api-tls\") pod \"maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld\" (UID: \"10015b7a-34e1-4c8a-be41-f9c63e942cfe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"
Aug 05 18:29:19.210910 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:19.210886    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59xtg\" (UniqueName: \"kubernetes.io/projected/10015b7a-34e1-4c8a-be41-f9c63e942cfe-kube-api-access-59xtg\") pod \"maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld\" (UID: \"10015b7a-34e1-4c8a-be41-f9c63e942cfe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"
Aug 05 18:29:19.220027 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:19.220007    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:29:19.308361 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:19.308288    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"
Aug 05 18:29:19.460897 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:19.460864    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"]
Aug 05 18:29:19.462419 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:29:19.462387    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10015b7a_34e1_4c8a_be41_f9c63e942cfe.slice/crio-c51ae4dac0390b31994e24437635935fba56088f0ddce649781cfa15f330eadb WatchSource:0}: Error finding container c51ae4dac0390b31994e24437635935fba56088f0ddce649781cfa15f330eadb: Status 404 returned error can't find the container with id c51ae4dac0390b31994e24437635935fba56088f0ddce649781cfa15f330eadb
Aug 05 18:29:19.464287 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:19.464254    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 18:29:19.512529 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:19.512487    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld" event={"ID":"10015b7a-34e1-4c8a-be41-f9c63e942cfe","Type":"ContainerStarted","Data":"c51ae4dac0390b31994e24437635935fba56088f0ddce649781cfa15f330eadb"}
Aug 05 18:29:19.553135 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:19.553078    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:29:19.651442 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:19.651394    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:29:20.518887 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:20.518854    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld" event={"ID":"10015b7a-34e1-4c8a-be41-f9c63e942cfe","Type":"ContainerStarted","Data":"f1cba52cbc8cb58f639e3abeb41513abe8eaea36115ec29fb294a775db4490e2"}
Aug 05 18:29:20.519320 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:20.518960    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"
Aug 05 18:29:20.541879 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:20.541830    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld" podStartSLOduration=2.06146839 podStartE2EDuration="2.541816446s" totalImagesPullingTime="480.348056ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:29:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:29:20.540000428 +0000 UTC m=+3367.892675023" watchObservedRunningTime="2026-08-05 18:29:20.541816446 +0000 UTC m=+3367.894491040"
Aug 05 18:29:26.529106 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:26.529074    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"
Aug 05 18:29:33.537901 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:33.537867    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:29:42.842143 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:42.842101    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:29:49.594240 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:49.594201    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-6tnnr_7b205b96-927c-4170-a375-5190000f495a/payload-processing/0.log"
Aug 05 18:29:58.632849 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.632819    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"]
Aug 05 18:29:58.636416 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.636401    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:58.639170 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.639148    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-b5b093\"/\"kube-root-ca.crt\""
Aug 05 18:29:58.640339 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.640321    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-b5b093\"/\"openshift-service-ca.crt\""
Aug 05 18:29:58.640438 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.640328    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-b5b093\"/\"ipp-route-b5b093-kserve-self-signed-certs\""
Aug 05 18:29:58.640438 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.640402    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-b5b093\"/\"default-dockercfg-fsw7d\""
Aug 05 18:29:58.645140 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.644924    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"]
Aug 05 18:29:58.660223 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.660195    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:29:58.743140 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.743113    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-home\") pod \"ipp-route-b5b093-kserve-56dc9665f-8j8bw\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") " pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:58.743140 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.743140    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-tmp-dir\") pod \"ipp-route-b5b093-kserve-56dc9665f-8j8bw\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") " pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:58.743349 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.743165    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-tls-certs\") pod \"ipp-route-b5b093-kserve-56dc9665f-8j8bw\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") " pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:58.743349 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.743201    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-model-cache\") pod \"ipp-route-b5b093-kserve-56dc9665f-8j8bw\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") " pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:58.743349 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.743332    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-dshm\") pod \"ipp-route-b5b093-kserve-56dc9665f-8j8bw\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") " pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:58.743464 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.743362    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbsls\" (UniqueName: \"kubernetes.io/projected/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-kube-api-access-fbsls\") pod \"ipp-route-b5b093-kserve-56dc9665f-8j8bw\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") " pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:58.844624 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.844595    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-dshm\") pod \"ipp-route-b5b093-kserve-56dc9665f-8j8bw\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") " pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:58.844739 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.844634    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbsls\" (UniqueName: \"kubernetes.io/projected/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-kube-api-access-fbsls\") pod \"ipp-route-b5b093-kserve-56dc9665f-8j8bw\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") " pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:58.844739 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.844684    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-home\") pod \"ipp-route-b5b093-kserve-56dc9665f-8j8bw\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") " pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:58.844739 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.844701    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-tmp-dir\") pod \"ipp-route-b5b093-kserve-56dc9665f-8j8bw\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") " pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:58.844739 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.844728    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-tls-certs\") pod \"ipp-route-b5b093-kserve-56dc9665f-8j8bw\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") " pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:58.844943 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.844815    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-model-cache\") pod \"ipp-route-b5b093-kserve-56dc9665f-8j8bw\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") " pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:58.845085 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.845062    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-tmp-dir\") pod \"ipp-route-b5b093-kserve-56dc9665f-8j8bw\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") " pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:58.845166 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.845096    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-home\") pod \"ipp-route-b5b093-kserve-56dc9665f-8j8bw\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") " pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:58.845254 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.845234    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-model-cache\") pod \"ipp-route-b5b093-kserve-56dc9665f-8j8bw\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") " pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:58.847014 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.846992    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-dshm\") pod \"ipp-route-b5b093-kserve-56dc9665f-8j8bw\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") " pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:58.847288 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.847270    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-tls-certs\") pod \"ipp-route-b5b093-kserve-56dc9665f-8j8bw\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") " pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:58.854384 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.854366    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbsls\" (UniqueName: \"kubernetes.io/projected/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-kube-api-access-fbsls\") pod \"ipp-route-b5b093-kserve-56dc9665f-8j8bw\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") " pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:58.947504 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:58.947449    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:29:59.087481 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:59.087458    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"]
Aug 05 18:29:59.089337 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:29:59.089305    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2348a2eb_64b8_4e73_8f2a_7e0aacbf8399.slice/crio-89052686b3953cc9d15e257a5cbf1e1d7aa30a89868de38d3b40742d15c90f6d WatchSource:0}: Error finding container 89052686b3953cc9d15e257a5cbf1e1d7aa30a89868de38d3b40742d15c90f6d: Status 404 returned error can't find the container with id 89052686b3953cc9d15e257a5cbf1e1d7aa30a89868de38d3b40742d15c90f6d
Aug 05 18:29:59.732729 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:59.732694    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw" event={"ID":"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399","Type":"ContainerStarted","Data":"f64c088a819207c0c63c42f9312a0b5924e5534a91a557568d43a240f7d489f5"}
Aug 05 18:29:59.732729 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:59.732734    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw" event={"ID":"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399","Type":"ContainerStarted","Data":"89052686b3953cc9d15e257a5cbf1e1d7aa30a89868de38d3b40742d15c90f6d"}
Aug 05 18:29:59.754356 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:29:59.754312    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw" podStartSLOduration=1.754297513 podStartE2EDuration="1.754297513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:29:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:29:59.752273602 +0000 UTC m=+3407.104948197" watchObservedRunningTime="2026-08-05 18:29:59.754297513 +0000 UTC m=+3407.106972107"
Aug 05 18:30:08.948398 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:08.948310    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:30:08.948815 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:08.948536    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:30:08.960836 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:08.960815    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:30:09.800037 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:09.800006    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:30:34.469383 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:34.469352    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc_dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db/payload-processing/0.log"
Aug 05 18:30:34.609787 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:34.609759    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-6tnnr_7b205b96-927c-4170-a375-5190000f495a/payload-processing/0.log"
Aug 05 18:30:35.042159 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:35.042126    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:30:35.366634 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:35.366604    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:30:35.688441 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:35.688354    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:30:38.854324 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:38.854289    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:30:38.948931 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:38.948900    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:30:39.937017 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:39.936972    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"]
Aug 05 18:30:39.940970 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:39.940946    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"
Aug 05 18:30:39.943854 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:39.943830    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-eef9c6-serving-cert\""
Aug 05 18:30:39.954915 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:39.954886    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"]
Aug 05 18:30:39.986015 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:39.985979    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:30:40.010448 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:40.010415    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ebad985-3d98-447d-ad95-a868108e610c-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh\" (UID: \"1ebad985-3d98-447d-ad95-a868108e610c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"
Aug 05 18:30:40.010592 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:40.010478    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ebad985-3d98-447d-ad95-a868108e610c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh\" (UID: \"1ebad985-3d98-447d-ad95-a868108e610c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"
Aug 05 18:30:40.010592 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:40.010508    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zhmh\" (UniqueName: \"kubernetes.io/projected/1ebad985-3d98-447d-ad95-a868108e610c-kube-api-access-9zhmh\") pod \"maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh\" (UID: \"1ebad985-3d98-447d-ad95-a868108e610c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"
Aug 05 18:30:40.083885 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:40.083857    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:30:40.111675 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:40.111648    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ebad985-3d98-447d-ad95-a868108e610c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh\" (UID: \"1ebad985-3d98-447d-ad95-a868108e610c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"
Aug 05 18:30:40.111772 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:40.111682    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zhmh\" (UniqueName: \"kubernetes.io/projected/1ebad985-3d98-447d-ad95-a868108e610c-kube-api-access-9zhmh\") pod \"maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh\" (UID: \"1ebad985-3d98-447d-ad95-a868108e610c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"
Aug 05 18:30:40.111832 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:40.111817    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ebad985-3d98-447d-ad95-a868108e610c-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh\" (UID: \"1ebad985-3d98-447d-ad95-a868108e610c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"
Aug 05 18:30:40.112464 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:40.112408    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ebad985-3d98-447d-ad95-a868108e610c-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh\" (UID: \"1ebad985-3d98-447d-ad95-a868108e610c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"
Aug 05 18:30:40.114277 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:40.114257    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ebad985-3d98-447d-ad95-a868108e610c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh\" (UID: \"1ebad985-3d98-447d-ad95-a868108e610c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"
Aug 05 18:30:40.120776 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:40.120751    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zhmh\" (UniqueName: \"kubernetes.io/projected/1ebad985-3d98-447d-ad95-a868108e610c-kube-api-access-9zhmh\") pod \"maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh\" (UID: \"1ebad985-3d98-447d-ad95-a868108e610c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"
Aug 05 18:30:40.254269 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:40.254205    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"
Aug 05 18:30:40.408022 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:40.407994    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"]
Aug 05 18:30:40.410065 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:30:40.410037    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ebad985_3d98_447d_ad95_a868108e610c.slice/crio-c32e34edec16bfc78befbfae90cbac103f9f04caa2df089d88226a10ab42b61e WatchSource:0}: Error finding container c32e34edec16bfc78befbfae90cbac103f9f04caa2df089d88226a10ab42b61e: Status 404 returned error can't find the container with id c32e34edec16bfc78befbfae90cbac103f9f04caa2df089d88226a10ab42b61e
Aug 05 18:30:40.506595 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:40.506522    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:30:40.608052 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:40.608018    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:30:40.969236 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:40.969204    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh" event={"ID":"1ebad985-3d98-447d-ad95-a868108e610c","Type":"ContainerStarted","Data":"f7464e52b2fe4ef0dca0afd6cb3c50e4bb1cf90cc4386942737c244b25a6d72a"}
Aug 05 18:30:40.969647 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:40.969259    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh" event={"ID":"1ebad985-3d98-447d-ad95-a868108e610c","Type":"ContainerStarted","Data":"c32e34edec16bfc78befbfae90cbac103f9f04caa2df089d88226a10ab42b61e"}
Aug 05 18:30:40.969647 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:40.969337    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"
Aug 05 18:30:40.989581 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:40.989537    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh" podStartSLOduration=1.644991133 podStartE2EDuration="1.989522655s" totalImagesPullingTime="344.531522ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:30:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:30:40.988343123 +0000 UTC m=+3448.341017738" watchObservedRunningTime="2026-08-05 18:30:40.989522655 +0000 UTC m=+3448.342197249"
Aug 05 18:30:46.979753 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:30:46.979724    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"
Aug 05 18:31:03.807769 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:03.807739    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:31:08.646752 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:08.646712    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:31:10.546773 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:10.546742    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"]
Aug 05 18:31:10.547240 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:10.546966    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh" podUID="1ebad985-3d98-447d-ad95-a868108e610c" containerName="maas-api" containerID="cri-o://f7464e52b2fe4ef0dca0afd6cb3c50e4bb1cf90cc4386942737c244b25a6d72a" gracePeriod=30
Aug 05 18:31:10.557921 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:10.557895    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:31:10.639556 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:10.639527    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:31:10.697386 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:10.697338    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:31:10.764118 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:10.764084    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:31:10.798443 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:10.798383    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"
Aug 05 18:31:10.898204 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:10.898163    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ebad985-3d98-447d-ad95-a868108e610c-openshift-service-ca\") pod \"1ebad985-3d98-447d-ad95-a868108e610c\" (UID: \"1ebad985-3d98-447d-ad95-a868108e610c\") "
Aug 05 18:31:10.898367 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:10.898230    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ebad985-3d98-447d-ad95-a868108e610c-maas-api-tls\") pod \"1ebad985-3d98-447d-ad95-a868108e610c\" (UID: \"1ebad985-3d98-447d-ad95-a868108e610c\") "
Aug 05 18:31:10.898367 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:10.898322    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zhmh\" (UniqueName: \"kubernetes.io/projected/1ebad985-3d98-447d-ad95-a868108e610c-kube-api-access-9zhmh\") pod \"1ebad985-3d98-447d-ad95-a868108e610c\" (UID: \"1ebad985-3d98-447d-ad95-a868108e610c\") "
Aug 05 18:31:10.898572 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:10.898544    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ebad985-3d98-447d-ad95-a868108e610c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1ebad985-3d98-447d-ad95-a868108e610c" (UID: "1ebad985-3d98-447d-ad95-a868108e610c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:31:10.898702 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:10.898658    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ebad985-3d98-447d-ad95-a868108e610c-openshift-service-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:31:10.900591 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:10.900560    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ebad985-3d98-447d-ad95-a868108e610c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1ebad985-3d98-447d-ad95-a868108e610c" (UID: "1ebad985-3d98-447d-ad95-a868108e610c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:31:10.900672 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:10.900566    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ebad985-3d98-447d-ad95-a868108e610c-kube-api-access-9zhmh" (OuterVolumeSpecName: "kube-api-access-9zhmh") pod "1ebad985-3d98-447d-ad95-a868108e610c" (UID: "1ebad985-3d98-447d-ad95-a868108e610c"). InnerVolumeSpecName "kube-api-access-9zhmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:31:10.999527 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:10.999498    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zhmh\" (UniqueName: \"kubernetes.io/projected/1ebad985-3d98-447d-ad95-a868108e610c-kube-api-access-9zhmh\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:31:10.999527 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:10.999522    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ebad985-3d98-447d-ad95-a868108e610c-maas-api-tls\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:31:11.093812 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:11.093775    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:31:11.132199 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:11.132154    2569 generic.go:356] "Generic (PLEG): container finished" podID="1ebad985-3d98-447d-ad95-a868108e610c" containerID="f7464e52b2fe4ef0dca0afd6cb3c50e4bb1cf90cc4386942737c244b25a6d72a" exitCode=0
Aug 05 18:31:11.132347 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:11.132220    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"
Aug 05 18:31:11.132347 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:11.132227    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh" event={"ID":"1ebad985-3d98-447d-ad95-a868108e610c","Type":"ContainerDied","Data":"f7464e52b2fe4ef0dca0afd6cb3c50e4bb1cf90cc4386942737c244b25a6d72a"}
Aug 05 18:31:11.132347 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:11.132270    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh" event={"ID":"1ebad985-3d98-447d-ad95-a868108e610c","Type":"ContainerDied","Data":"c32e34edec16bfc78befbfae90cbac103f9f04caa2df089d88226a10ab42b61e"}
Aug 05 18:31:11.132347 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:11.132294    2569 scope.go:117] "RemoveContainer" containerID="f7464e52b2fe4ef0dca0afd6cb3c50e4bb1cf90cc4386942737c244b25a6d72a"
Aug 05 18:31:11.141957 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:11.141939    2569 scope.go:117] "RemoveContainer" containerID="f7464e52b2fe4ef0dca0afd6cb3c50e4bb1cf90cc4386942737c244b25a6d72a"
Aug 05 18:31:11.142206 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:31:11.142168    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7464e52b2fe4ef0dca0afd6cb3c50e4bb1cf90cc4386942737c244b25a6d72a\": container with ID starting with f7464e52b2fe4ef0dca0afd6cb3c50e4bb1cf90cc4386942737c244b25a6d72a not found: ID does not exist" containerID="f7464e52b2fe4ef0dca0afd6cb3c50e4bb1cf90cc4386942737c244b25a6d72a"
Aug 05 18:31:11.142279 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:11.142214    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7464e52b2fe4ef0dca0afd6cb3c50e4bb1cf90cc4386942737c244b25a6d72a"} err="failed to get container status \"f7464e52b2fe4ef0dca0afd6cb3c50e4bb1cf90cc4386942737c244b25a6d72a\": rpc error: code = NotFound desc = could not find container \"f7464e52b2fe4ef0dca0afd6cb3c50e4bb1cf90cc4386942737c244b25a6d72a\": container with ID starting with f7464e52b2fe4ef0dca0afd6cb3c50e4bb1cf90cc4386942737c244b25a6d72a not found: ID does not exist"
Aug 05 18:31:11.154557 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:11.154534    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"]
Aug 05 18:31:11.161437 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:11.161418    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eef9c6-5f65747445-jcxnh"]
Aug 05 18:31:13.152429 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:13.152390    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ebad985-3d98-447d-ad95-a868108e610c" path="/var/lib/kubelet/pods/1ebad985-3d98-447d-ad95-a868108e610c/volumes"
Aug 05 18:31:17.257368 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:17.257327    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:31:17.362238 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:17.362208    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:31:17.564962 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:17.564868    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:31:17.773329 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:17.773296    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:31:36.661795 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:36.661593    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc"]
Aug 05 18:31:36.662512 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:36.662447    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc" podUID="dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db" containerName="payload-processing" containerID="cri-o://e9bd87381b6b43f5f0696954fba22376f24f2f40b3e46ff8fcfa577aa8e1ace1" gracePeriod=30
Aug 05 18:31:36.673037 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:36.672968    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:31:36.762444 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:36.762415    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:31:36.860809 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:36.860711    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:31:36.921771 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:36.921718    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc"
Aug 05 18:31:37.025243 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:37.025216    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db-plugins-config-volume\") pod \"dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db\" (UID: \"dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db\") "
Aug 05 18:31:37.025389 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:37.025308    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9cf7f\" (UniqueName: \"kubernetes.io/projected/dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db-kube-api-access-9cf7f\") pod \"dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db\" (UID: \"dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db\") "
Aug 05 18:31:37.025716 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:37.025689    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db" (UID: "dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:31:37.027560 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:37.027530    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db-kube-api-access-9cf7f" (OuterVolumeSpecName: "kube-api-access-9cf7f") pod "dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db" (UID: "dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db"). InnerVolumeSpecName "kube-api-access-9cf7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:31:37.126210 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:37.126166    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:31:37.126210 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:37.126208    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cf7f\" (UniqueName: \"kubernetes.io/projected/dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db-kube-api-access-9cf7f\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:31:37.187001 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:37.186940    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:31:37.270819 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:37.270793    2569 generic.go:356] "Generic (PLEG): container finished" podID="dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db" containerID="e9bd87381b6b43f5f0696954fba22376f24f2f40b3e46ff8fcfa577aa8e1ace1" exitCode=0
Aug 05 18:31:37.270912 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:37.270829    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc"
Aug 05 18:31:37.270912 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:37.270834    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc" event={"ID":"dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db","Type":"ContainerDied","Data":"e9bd87381b6b43f5f0696954fba22376f24f2f40b3e46ff8fcfa577aa8e1ace1"}
Aug 05 18:31:37.270912 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:37.270881    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc" event={"ID":"dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db","Type":"ContainerDied","Data":"898430b817012499db184faeb4f2aca8fb7d53d692061843c0987f33c016a8a4"}
Aug 05 18:31:37.270912 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:37.270902    2569 scope.go:117] "RemoveContainer" containerID="e9bd87381b6b43f5f0696954fba22376f24f2f40b3e46ff8fcfa577aa8e1ace1"
Aug 05 18:31:37.280711 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:37.280696    2569 scope.go:117] "RemoveContainer" containerID="e9bd87381b6b43f5f0696954fba22376f24f2f40b3e46ff8fcfa577aa8e1ace1"
Aug 05 18:31:37.280959 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:31:37.280940    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9bd87381b6b43f5f0696954fba22376f24f2f40b3e46ff8fcfa577aa8e1ace1\": container with ID starting with e9bd87381b6b43f5f0696954fba22376f24f2f40b3e46ff8fcfa577aa8e1ace1 not found: ID does not exist" containerID="e9bd87381b6b43f5f0696954fba22376f24f2f40b3e46ff8fcfa577aa8e1ace1"
Aug 05 18:31:37.281022 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:37.280976    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9bd87381b6b43f5f0696954fba22376f24f2f40b3e46ff8fcfa577aa8e1ace1"} err="failed to get container status \"e9bd87381b6b43f5f0696954fba22376f24f2f40b3e46ff8fcfa577aa8e1ace1\": rpc error: code = NotFound desc = could not find container \"e9bd87381b6b43f5f0696954fba22376f24f2f40b3e46ff8fcfa577aa8e1ace1\": container with ID starting with e9bd87381b6b43f5f0696954fba22376f24f2f40b3e46ff8fcfa577aa8e1ace1 not found: ID does not exist"
Aug 05 18:31:37.288377 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:37.288357    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc"]
Aug 05 18:31:37.294346 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:37.294326    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b5b093-5f5f56fc9b-72lrc"]
Aug 05 18:31:39.152704 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:39.152671    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db" path="/var/lib/kubelet/pods/dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db/volumes"
Aug 05 18:31:42.193867 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:42.193837    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"]
Aug 05 18:31:42.312061 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:42.311992    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:31:42.586438 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:42.586373    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:31:57.213700 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.213662    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw" podUID="2348a2eb-64b8-4e73-8f2a-7e0aacbf8399" containerName="main" containerID="cri-o://f64c088a819207c0c63c42f9312a0b5924e5534a91a557568d43a240f7d489f5" gracePeriod=45
Aug 05 18:31:57.374549 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.374518    2569 generic.go:356] "Generic (PLEG): container finished" podID="2348a2eb-64b8-4e73-8f2a-7e0aacbf8399" containerID="f64c088a819207c0c63c42f9312a0b5924e5534a91a557568d43a240f7d489f5" exitCode=0
Aug 05 18:31:57.374692 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.374590    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw" event={"ID":"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399","Type":"ContainerDied","Data":"f64c088a819207c0c63c42f9312a0b5924e5534a91a557568d43a240f7d489f5"}
Aug 05 18:31:57.473748 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.473689    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:31:57.597092 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.597058    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-dshm\") pod \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") "
Aug 05 18:31:57.597278 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.597102    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-tmp-dir\") pod \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") "
Aug 05 18:31:57.597278 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.597208    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbsls\" (UniqueName: \"kubernetes.io/projected/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-kube-api-access-fbsls\") pod \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") "
Aug 05 18:31:57.597383 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.597288    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-model-cache\") pod \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") "
Aug 05 18:31:57.597383 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.597322    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-home\") pod \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") "
Aug 05 18:31:57.597383 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.597351    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-tls-certs\") pod \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\" (UID: \"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399\") "
Aug 05 18:31:57.597539 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.597410    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2348a2eb-64b8-4e73-8f2a-7e0aacbf8399" (UID: "2348a2eb-64b8-4e73-8f2a-7e0aacbf8399"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:31:57.597628 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.597605    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-model-cache" (OuterVolumeSpecName: "model-cache") pod "2348a2eb-64b8-4e73-8f2a-7e0aacbf8399" (UID: "2348a2eb-64b8-4e73-8f2a-7e0aacbf8399"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:31:57.597713 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.597622    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-home" (OuterVolumeSpecName: "home") pod "2348a2eb-64b8-4e73-8f2a-7e0aacbf8399" (UID: "2348a2eb-64b8-4e73-8f2a-7e0aacbf8399"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:31:57.597768 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.597756    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-tmp-dir\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:31:57.597822 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.597776    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-model-cache\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:31:57.597822 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.597790    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-home\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:31:57.599489 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.599462    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-dshm" (OuterVolumeSpecName: "dshm") pod "2348a2eb-64b8-4e73-8f2a-7e0aacbf8399" (UID: "2348a2eb-64b8-4e73-8f2a-7e0aacbf8399"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:31:57.599624 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.599605    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-kube-api-access-fbsls" (OuterVolumeSpecName: "kube-api-access-fbsls") pod "2348a2eb-64b8-4e73-8f2a-7e0aacbf8399" (UID: "2348a2eb-64b8-4e73-8f2a-7e0aacbf8399"). InnerVolumeSpecName "kube-api-access-fbsls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:31:57.599820 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.599797    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2348a2eb-64b8-4e73-8f2a-7e0aacbf8399" (UID: "2348a2eb-64b8-4e73-8f2a-7e0aacbf8399"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:31:57.698602 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.698578    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-dshm\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:31:57.698602 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.698597    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbsls\" (UniqueName: \"kubernetes.io/projected/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-kube-api-access-fbsls\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:31:57.698754 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:57.698608    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399-tls-certs\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:31:58.381801 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:58.381763    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw" event={"ID":"2348a2eb-64b8-4e73-8f2a-7e0aacbf8399","Type":"ContainerDied","Data":"89052686b3953cc9d15e257a5cbf1e1d7aa30a89868de38d3b40742d15c90f6d"}
Aug 05 18:31:58.381801 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:58.381786    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"
Aug 05 18:31:58.382365 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:58.381821    2569 scope.go:117] "RemoveContainer" containerID="f64c088a819207c0c63c42f9312a0b5924e5534a91a557568d43a240f7d489f5"
Aug 05 18:31:58.404114 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:58.404090    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"]
Aug 05 18:31:58.408453 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:58.408432    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-b5b093/ipp-route-b5b093-kserve-56dc9665f-8j8bw"]
Aug 05 18:31:59.151444 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:31:59.151412    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2348a2eb-64b8-4e73-8f2a-7e0aacbf8399" path="/var/lib/kubelet/pods/2348a2eb-64b8-4e73-8f2a-7e0aacbf8399/volumes"
Aug 05 18:32:18.408076 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:18.408039    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:20.624857 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:20.624804    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:26.820485 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:26.820450    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:26.892816 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:26.892776    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:27.122078 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:27.121993    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:27.342648 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:27.342618    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:29.947381 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:29.947354    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:30.044419 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:30.044390    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:37.991575 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:37.991546    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"]
Aug 05 18:32:37.991977 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:37.991752    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld" podUID="10015b7a-34e1-4c8a-be41-f9c63e942cfe" containerName="maas-api" containerID="cri-o://f1cba52cbc8cb58f639e3abeb41513abe8eaea36115ec29fb294a775db4490e2" gracePeriod=30
Aug 05 18:32:38.019889 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.019863    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:38.120681 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.120652    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:38.211664 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.211633    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:38.247428 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.247374    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"
Aug 05 18:32:38.341460 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.341438    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10015b7a-34e1-4c8a-be41-f9c63e942cfe-maas-api-tls\") pod \"10015b7a-34e1-4c8a-be41-f9c63e942cfe\" (UID: \"10015b7a-34e1-4c8a-be41-f9c63e942cfe\") "
Aug 05 18:32:38.341600 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.341504    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10015b7a-34e1-4c8a-be41-f9c63e942cfe-openshift-service-ca\") pod \"10015b7a-34e1-4c8a-be41-f9c63e942cfe\" (UID: \"10015b7a-34e1-4c8a-be41-f9c63e942cfe\") "
Aug 05 18:32:38.341652 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.341600    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59xtg\" (UniqueName: \"kubernetes.io/projected/10015b7a-34e1-4c8a-be41-f9c63e942cfe-kube-api-access-59xtg\") pod \"10015b7a-34e1-4c8a-be41-f9c63e942cfe\" (UID: \"10015b7a-34e1-4c8a-be41-f9c63e942cfe\") "
Aug 05 18:32:38.341777 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.341760    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10015b7a-34e1-4c8a-be41-f9c63e942cfe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "10015b7a-34e1-4c8a-be41-f9c63e942cfe" (UID: "10015b7a-34e1-4c8a-be41-f9c63e942cfe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:32:38.341894 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.341881    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10015b7a-34e1-4c8a-be41-f9c63e942cfe-openshift-service-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:32:38.343550 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.343525    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10015b7a-34e1-4c8a-be41-f9c63e942cfe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "10015b7a-34e1-4c8a-be41-f9c63e942cfe" (UID: "10015b7a-34e1-4c8a-be41-f9c63e942cfe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:32:38.343691 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.343672    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10015b7a-34e1-4c8a-be41-f9c63e942cfe-kube-api-access-59xtg" (OuterVolumeSpecName: "kube-api-access-59xtg") pod "10015b7a-34e1-4c8a-be41-f9c63e942cfe" (UID: "10015b7a-34e1-4c8a-be41-f9c63e942cfe"). InnerVolumeSpecName "kube-api-access-59xtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:32:38.442868 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.442839    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59xtg\" (UniqueName: \"kubernetes.io/projected/10015b7a-34e1-4c8a-be41-f9c63e942cfe-kube-api-access-59xtg\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:32:38.442868 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.442864    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10015b7a-34e1-4c8a-be41-f9c63e942cfe-maas-api-tls\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:32:38.572234 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.572207    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:38.588566 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.588543    2569 generic.go:356] "Generic (PLEG): container finished" podID="10015b7a-34e1-4c8a-be41-f9c63e942cfe" containerID="f1cba52cbc8cb58f639e3abeb41513abe8eaea36115ec29fb294a775db4490e2" exitCode=0
Aug 05 18:32:38.588669 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.588582    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"
Aug 05 18:32:38.588669 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.588626    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld" event={"ID":"10015b7a-34e1-4c8a-be41-f9c63e942cfe","Type":"ContainerDied","Data":"f1cba52cbc8cb58f639e3abeb41513abe8eaea36115ec29fb294a775db4490e2"}
Aug 05 18:32:38.588759 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.588677    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld" event={"ID":"10015b7a-34e1-4c8a-be41-f9c63e942cfe","Type":"ContainerDied","Data":"c51ae4dac0390b31994e24437635935fba56088f0ddce649781cfa15f330eadb"}
Aug 05 18:32:38.588759 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.588700    2569 scope.go:117] "RemoveContainer" containerID="f1cba52cbc8cb58f639e3abeb41513abe8eaea36115ec29fb294a775db4490e2"
Aug 05 18:32:38.598382 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.598367    2569 scope.go:117] "RemoveContainer" containerID="f1cba52cbc8cb58f639e3abeb41513abe8eaea36115ec29fb294a775db4490e2"
Aug 05 18:32:38.598625 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:32:38.598603    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1cba52cbc8cb58f639e3abeb41513abe8eaea36115ec29fb294a775db4490e2\": container with ID starting with f1cba52cbc8cb58f639e3abeb41513abe8eaea36115ec29fb294a775db4490e2 not found: ID does not exist" containerID="f1cba52cbc8cb58f639e3abeb41513abe8eaea36115ec29fb294a775db4490e2"
Aug 05 18:32:38.598725 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.598631    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1cba52cbc8cb58f639e3abeb41513abe8eaea36115ec29fb294a775db4490e2"} err="failed to get container status \"f1cba52cbc8cb58f639e3abeb41513abe8eaea36115ec29fb294a775db4490e2\": rpc error: code = NotFound desc = could not find container \"f1cba52cbc8cb58f639e3abeb41513abe8eaea36115ec29fb294a775db4490e2\": container with ID starting with f1cba52cbc8cb58f639e3abeb41513abe8eaea36115ec29fb294a775db4490e2 not found: ID does not exist"
Aug 05 18:32:38.627473 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.627443    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"]
Aug 05 18:32:38.630690 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:38.630669    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cea6f9-6f46594cbc-m5nld"]
Aug 05 18:32:39.150763 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:39.150732    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10015b7a-34e1-4c8a-be41-f9c63e942cfe" path="/var/lib/kubelet/pods/10015b7a-34e1-4c8a-be41-f9c63e942cfe/volumes"
Aug 05 18:32:45.155607 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:45.155571    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:45.240398 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:45.240362    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:45.443896 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:45.443805    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:45.664515 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:45.664475    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:56.282100 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.282065    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"]
Aug 05 18:32:56.282532 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.282294    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj" podUID="77e76780-cf13-4569-ba5a-43bcb5409418" containerName="maas-api" containerID="cri-o://245d816a71abf4f7f82b1cf0c6363a4ce016f32729320ec8c5678b56fa33e3fa" gracePeriod=30
Aug 05 18:32:56.330927 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.330903    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:56.351305 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.351211    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr"]
Aug 05 18:32:56.352045 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.351436    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr" podUID="e746138a-5050-4ec5-98db-304839bffc5c" containerName="payload-processing" containerID="cri-o://81ad8506828f2ef2f34e7bb36bb31b8302538fd998ee24861d50e8e43cb1766d" gracePeriod=30
Aug 05 18:32:56.354289 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.354262    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd"]
Aug 05 18:32:56.354490 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.354459    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd" podUID="f2473b44-6051-4d8e-b17a-349d85441ae6" containerName="payload-pre-processing" containerID="cri-o://c08fdeb715ebc17b9cd01bf38ca02de7d89e86fb750ea78c81255486242a41b8" gracePeriod=30
Aug 05 18:32:56.430578 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.430552    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:56.551453 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.551361    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:56.623094 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.623071    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"
Aug 05 18:32:56.649690 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.649655    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd"
Aug 05 18:32:56.664965 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.664942    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr"
Aug 05 18:32:56.683842 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.683820    2569 generic.go:356] "Generic (PLEG): container finished" podID="77e76780-cf13-4569-ba5a-43bcb5409418" containerID="245d816a71abf4f7f82b1cf0c6363a4ce016f32729320ec8c5678b56fa33e3fa" exitCode=0
Aug 05 18:32:56.683964 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.683868    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"
Aug 05 18:32:56.683964 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.683908    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj" event={"ID":"77e76780-cf13-4569-ba5a-43bcb5409418","Type":"ContainerDied","Data":"245d816a71abf4f7f82b1cf0c6363a4ce016f32729320ec8c5678b56fa33e3fa"}
Aug 05 18:32:56.683964 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.683960    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj" event={"ID":"77e76780-cf13-4569-ba5a-43bcb5409418","Type":"ContainerDied","Data":"6d046e296877926c93c5a9b43cf971f0f4b3806b19599bb069073dd2015d9b27"}
Aug 05 18:32:56.684091 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.683984    2569 scope.go:117] "RemoveContainer" containerID="245d816a71abf4f7f82b1cf0c6363a4ce016f32729320ec8c5678b56fa33e3fa"
Aug 05 18:32:56.685504 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.685480    2569 generic.go:356] "Generic (PLEG): container finished" podID="e746138a-5050-4ec5-98db-304839bffc5c" containerID="81ad8506828f2ef2f34e7bb36bb31b8302538fd998ee24861d50e8e43cb1766d" exitCode=0
Aug 05 18:32:56.685595 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.685511    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr"
Aug 05 18:32:56.685664 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.685592    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr" event={"ID":"e746138a-5050-4ec5-98db-304839bffc5c","Type":"ContainerDied","Data":"81ad8506828f2ef2f34e7bb36bb31b8302538fd998ee24861d50e8e43cb1766d"}
Aug 05 18:32:56.685664 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.685638    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr" event={"ID":"e746138a-5050-4ec5-98db-304839bffc5c","Type":"ContainerDied","Data":"584d20b978dbb731dfc664f078fc48a99d773d35cd932269ed3ebc69ee8d531f"}
Aug 05 18:32:56.686869 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.686850    2569 generic.go:356] "Generic (PLEG): container finished" podID="f2473b44-6051-4d8e-b17a-349d85441ae6" containerID="c08fdeb715ebc17b9cd01bf38ca02de7d89e86fb750ea78c81255486242a41b8" exitCode=0
Aug 05 18:32:56.686972 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.686881    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd" event={"ID":"f2473b44-6051-4d8e-b17a-349d85441ae6","Type":"ContainerDied","Data":"c08fdeb715ebc17b9cd01bf38ca02de7d89e86fb750ea78c81255486242a41b8"}
Aug 05 18:32:56.686972 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.686915    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd" event={"ID":"f2473b44-6051-4d8e-b17a-349d85441ae6","Type":"ContainerDied","Data":"408c240a3c29d790e4416548dbc56c92cbeb63dd86f03d821c8067df10807e49"}
Aug 05 18:32:56.686972 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.686886    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd"
Aug 05 18:32:56.693451 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.693424    2569 scope.go:117] "RemoveContainer" containerID="245d816a71abf4f7f82b1cf0c6363a4ce016f32729320ec8c5678b56fa33e3fa"
Aug 05 18:32:56.693710 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:32:56.693690    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"245d816a71abf4f7f82b1cf0c6363a4ce016f32729320ec8c5678b56fa33e3fa\": container with ID starting with 245d816a71abf4f7f82b1cf0c6363a4ce016f32729320ec8c5678b56fa33e3fa not found: ID does not exist" containerID="245d816a71abf4f7f82b1cf0c6363a4ce016f32729320ec8c5678b56fa33e3fa"
Aug 05 18:32:56.693767 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.693718    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"245d816a71abf4f7f82b1cf0c6363a4ce016f32729320ec8c5678b56fa33e3fa"} err="failed to get container status \"245d816a71abf4f7f82b1cf0c6363a4ce016f32729320ec8c5678b56fa33e3fa\": rpc error: code = NotFound desc = could not find container \"245d816a71abf4f7f82b1cf0c6363a4ce016f32729320ec8c5678b56fa33e3fa\": container with ID starting with 245d816a71abf4f7f82b1cf0c6363a4ce016f32729320ec8c5678b56fa33e3fa not found: ID does not exist"
Aug 05 18:32:56.693767 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.693733    2569 scope.go:117] "RemoveContainer" containerID="81ad8506828f2ef2f34e7bb36bb31b8302538fd998ee24861d50e8e43cb1766d"
Aug 05 18:32:56.710420 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.710402    2569 scope.go:117] "RemoveContainer" containerID="81ad8506828f2ef2f34e7bb36bb31b8302538fd998ee24861d50e8e43cb1766d"
Aug 05 18:32:56.711002 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:32:56.710980    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81ad8506828f2ef2f34e7bb36bb31b8302538fd998ee24861d50e8e43cb1766d\": container with ID starting with 81ad8506828f2ef2f34e7bb36bb31b8302538fd998ee24861d50e8e43cb1766d not found: ID does not exist" containerID="81ad8506828f2ef2f34e7bb36bb31b8302538fd998ee24861d50e8e43cb1766d"
Aug 05 18:32:56.711107 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.711007    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81ad8506828f2ef2f34e7bb36bb31b8302538fd998ee24861d50e8e43cb1766d"} err="failed to get container status \"81ad8506828f2ef2f34e7bb36bb31b8302538fd998ee24861d50e8e43cb1766d\": rpc error: code = NotFound desc = could not find container \"81ad8506828f2ef2f34e7bb36bb31b8302538fd998ee24861d50e8e43cb1766d\": container with ID starting with 81ad8506828f2ef2f34e7bb36bb31b8302538fd998ee24861d50e8e43cb1766d not found: ID does not exist"
Aug 05 18:32:56.711169 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.711150    2569 scope.go:117] "RemoveContainer" containerID="c08fdeb715ebc17b9cd01bf38ca02de7d89e86fb750ea78c81255486242a41b8"
Aug 05 18:32:56.719450 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.719432    2569 scope.go:117] "RemoveContainer" containerID="c08fdeb715ebc17b9cd01bf38ca02de7d89e86fb750ea78c81255486242a41b8"
Aug 05 18:32:56.719709 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:32:56.719689    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c08fdeb715ebc17b9cd01bf38ca02de7d89e86fb750ea78c81255486242a41b8\": container with ID starting with c08fdeb715ebc17b9cd01bf38ca02de7d89e86fb750ea78c81255486242a41b8 not found: ID does not exist" containerID="c08fdeb715ebc17b9cd01bf38ca02de7d89e86fb750ea78c81255486242a41b8"
Aug 05 18:32:56.719788 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.719716    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c08fdeb715ebc17b9cd01bf38ca02de7d89e86fb750ea78c81255486242a41b8"} err="failed to get container status \"c08fdeb715ebc17b9cd01bf38ca02de7d89e86fb750ea78c81255486242a41b8\": rpc error: code = NotFound desc = could not find container \"c08fdeb715ebc17b9cd01bf38ca02de7d89e86fb750ea78c81255486242a41b8\": container with ID starting with c08fdeb715ebc17b9cd01bf38ca02de7d89e86fb750ea78c81255486242a41b8 not found: ID does not exist"
Aug 05 18:32:56.801925 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.801872    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fbh2\" (UniqueName: \"kubernetes.io/projected/f2473b44-6051-4d8e-b17a-349d85441ae6-kube-api-access-7fbh2\") pod \"f2473b44-6051-4d8e-b17a-349d85441ae6\" (UID: \"f2473b44-6051-4d8e-b17a-349d85441ae6\") "
Aug 05 18:32:56.801925 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.801914    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvggr\" (UniqueName: \"kubernetes.io/projected/e746138a-5050-4ec5-98db-304839bffc5c-kube-api-access-gvggr\") pod \"e746138a-5050-4ec5-98db-304839bffc5c\" (UID: \"e746138a-5050-4ec5-98db-304839bffc5c\") "
Aug 05 18:32:56.802036 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.801966    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7drff\" (UniqueName: \"kubernetes.io/projected/77e76780-cf13-4569-ba5a-43bcb5409418-kube-api-access-7drff\") pod \"77e76780-cf13-4569-ba5a-43bcb5409418\" (UID: \"77e76780-cf13-4569-ba5a-43bcb5409418\") "
Aug 05 18:32:56.802036 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.802004    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e746138a-5050-4ec5-98db-304839bffc5c-plugins-config-volume\") pod \"e746138a-5050-4ec5-98db-304839bffc5c\" (UID: \"e746138a-5050-4ec5-98db-304839bffc5c\") "
Aug 05 18:32:56.802129 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.802043    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2473b44-6051-4d8e-b17a-349d85441ae6-plugins-config-volume\") pod \"f2473b44-6051-4d8e-b17a-349d85441ae6\" (UID: \"f2473b44-6051-4d8e-b17a-349d85441ae6\") "
Aug 05 18:32:56.802129 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.802073    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77e76780-cf13-4569-ba5a-43bcb5409418-openshift-service-ca\") pod \"77e76780-cf13-4569-ba5a-43bcb5409418\" (UID: \"77e76780-cf13-4569-ba5a-43bcb5409418\") "
Aug 05 18:32:56.802129 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.802097    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77e76780-cf13-4569-ba5a-43bcb5409418-maas-api-tls\") pod \"77e76780-cf13-4569-ba5a-43bcb5409418\" (UID: \"77e76780-cf13-4569-ba5a-43bcb5409418\") "
Aug 05 18:32:56.802484 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.802458    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2473b44-6051-4d8e-b17a-349d85441ae6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f2473b44-6051-4d8e-b17a-349d85441ae6" (UID: "f2473b44-6051-4d8e-b17a-349d85441ae6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:32:56.802618 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.802458    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e746138a-5050-4ec5-98db-304839bffc5c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e746138a-5050-4ec5-98db-304839bffc5c" (UID: "e746138a-5050-4ec5-98db-304839bffc5c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:32:56.802673 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.802615    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77e76780-cf13-4569-ba5a-43bcb5409418-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "77e76780-cf13-4569-ba5a-43bcb5409418" (UID: "77e76780-cf13-4569-ba5a-43bcb5409418"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:32:56.804137 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.804115    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77e76780-cf13-4569-ba5a-43bcb5409418-kube-api-access-7drff" (OuterVolumeSpecName: "kube-api-access-7drff") pod "77e76780-cf13-4569-ba5a-43bcb5409418" (UID: "77e76780-cf13-4569-ba5a-43bcb5409418"). InnerVolumeSpecName "kube-api-access-7drff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:32:56.804329 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.804265    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e746138a-5050-4ec5-98db-304839bffc5c-kube-api-access-gvggr" (OuterVolumeSpecName: "kube-api-access-gvggr") pod "e746138a-5050-4ec5-98db-304839bffc5c" (UID: "e746138a-5050-4ec5-98db-304839bffc5c"). InnerVolumeSpecName "kube-api-access-gvggr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:32:56.804647 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.804627    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77e76780-cf13-4569-ba5a-43bcb5409418-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "77e76780-cf13-4569-ba5a-43bcb5409418" (UID: "77e76780-cf13-4569-ba5a-43bcb5409418"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:32:56.804699 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.804629    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2473b44-6051-4d8e-b17a-349d85441ae6-kube-api-access-7fbh2" (OuterVolumeSpecName: "kube-api-access-7fbh2") pod "f2473b44-6051-4d8e-b17a-349d85441ae6" (UID: "f2473b44-6051-4d8e-b17a-349d85441ae6"). InnerVolumeSpecName "kube-api-access-7fbh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:32:56.903173 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.903148    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvggr\" (UniqueName: \"kubernetes.io/projected/e746138a-5050-4ec5-98db-304839bffc5c-kube-api-access-gvggr\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:32:56.903173 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.903170    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7drff\" (UniqueName: \"kubernetes.io/projected/77e76780-cf13-4569-ba5a-43bcb5409418-kube-api-access-7drff\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:32:56.903309 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.903200    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e746138a-5050-4ec5-98db-304839bffc5c-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:32:56.903309 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.903208    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2473b44-6051-4d8e-b17a-349d85441ae6-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:32:56.903309 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.903217    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77e76780-cf13-4569-ba5a-43bcb5409418-openshift-service-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:32:56.903309 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.903225    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77e76780-cf13-4569-ba5a-43bcb5409418-maas-api-tls\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:32:56.903309 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.903233    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fbh2\" (UniqueName: \"kubernetes.io/projected/f2473b44-6051-4d8e-b17a-349d85441ae6-kube-api-access-7fbh2\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:32:56.907553 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:56.907533    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:32:57.013082 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:57.013059    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd"]
Aug 05 18:32:57.018019 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:57.017999    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-70b4a4-557d7987b4-mknjd"]
Aug 05 18:32:57.030529 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:57.030504    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr"]
Aug 05 18:32:57.035132 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:57.035114    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-70b4a4-cbd6f675d-6t7rr"]
Aug 05 18:32:57.046737 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:57.046717    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"]
Aug 05 18:32:57.054195 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:57.054132    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-70b4a4-758d479996-4lkwj"]
Aug 05 18:32:57.150959 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:57.150936    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77e76780-cf13-4569-ba5a-43bcb5409418" path="/var/lib/kubelet/pods/77e76780-cf13-4569-ba5a-43bcb5409418/volumes"
Aug 05 18:32:57.151321 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:57.151307    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e746138a-5050-4ec5-98db-304839bffc5c" path="/var/lib/kubelet/pods/e746138a-5050-4ec5-98db-304839bffc5c/volumes"
Aug 05 18:32:57.151614 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:32:57.151603    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2473b44-6051-4d8e-b17a-349d85441ae6" path="/var/lib/kubelet/pods/f2473b44-6051-4d8e-b17a-349d85441ae6/volumes"
Aug 05 18:33:03.641321 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:03.641283    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:33:03.743749 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:03.743711    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:33:03.940667 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:03.940562    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:33:04.161844 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:04.161815    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:33:15.019172 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.019136    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"]
Aug 05 18:33:15.019781 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.019442    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq" podUID="7ca7e691-e4ad-48a3-b3d3-e17d4858e002" containerName="maas-api" containerID="cri-o://5cd664d124081b4ec25b0089f08439cf68ef0a516d4a5f0046b0a938f9bc4c0c" gracePeriod=30
Aug 05 18:33:15.062624 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.062594    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:33:15.070263 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.070230    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf"]
Aug 05 18:33:15.070469 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.070445    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf" podUID="133dc018-7d7b-463d-bbb9-d4e0cecfa306" containerName="payload-pre-processing" containerID="cri-o://2a463a8da7c41d026dc43dbbab7fc0fb713d40c8739aa6aece18d2682d4a6821" gracePeriod=30
Aug 05 18:33:15.168483 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.168453    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:33:15.258559 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.258458    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:33:15.323755 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.323725    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"
Aug 05 18:33:15.354058 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.354036    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf"
Aug 05 18:33:15.358169 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.358144    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tqmt\" (UniqueName: \"kubernetes.io/projected/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-kube-api-access-7tqmt\") pod \"7ca7e691-e4ad-48a3-b3d3-e17d4858e002\" (UID: \"7ca7e691-e4ad-48a3-b3d3-e17d4858e002\") "
Aug 05 18:33:15.358324 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.358229    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-openshift-service-ca\") pod \"7ca7e691-e4ad-48a3-b3d3-e17d4858e002\" (UID: \"7ca7e691-e4ad-48a3-b3d3-e17d4858e002\") "
Aug 05 18:33:15.358324 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.358268    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-maas-api-tls\") pod \"7ca7e691-e4ad-48a3-b3d3-e17d4858e002\" (UID: \"7ca7e691-e4ad-48a3-b3d3-e17d4858e002\") "
Aug 05 18:33:15.358558 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.358524    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7ca7e691-e4ad-48a3-b3d3-e17d4858e002" (UID: "7ca7e691-e4ad-48a3-b3d3-e17d4858e002"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:33:15.360445 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.360422    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7ca7e691-e4ad-48a3-b3d3-e17d4858e002" (UID: "7ca7e691-e4ad-48a3-b3d3-e17d4858e002"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:33:15.360520 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.360442    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-kube-api-access-7tqmt" (OuterVolumeSpecName: "kube-api-access-7tqmt") pod "7ca7e691-e4ad-48a3-b3d3-e17d4858e002" (UID: "7ca7e691-e4ad-48a3-b3d3-e17d4858e002"). InnerVolumeSpecName "kube-api-access-7tqmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:33:15.459744 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.459709    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7t29j\" (UniqueName: \"kubernetes.io/projected/133dc018-7d7b-463d-bbb9-d4e0cecfa306-kube-api-access-7t29j\") pod \"133dc018-7d7b-463d-bbb9-d4e0cecfa306\" (UID: \"133dc018-7d7b-463d-bbb9-d4e0cecfa306\") "
Aug 05 18:33:15.459911 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.459759    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/133dc018-7d7b-463d-bbb9-d4e0cecfa306-plugins-config-volume\") pod \"133dc018-7d7b-463d-bbb9-d4e0cecfa306\" (UID: \"133dc018-7d7b-463d-bbb9-d4e0cecfa306\") "
Aug 05 18:33:15.459961 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.459933    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tqmt\" (UniqueName: \"kubernetes.io/projected/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-kube-api-access-7tqmt\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:33:15.459961 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.459944    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-openshift-service-ca\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:33:15.459961 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.459954    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ca7e691-e4ad-48a3-b3d3-e17d4858e002-maas-api-tls\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:33:15.460141 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.460121    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/133dc018-7d7b-463d-bbb9-d4e0cecfa306-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "133dc018-7d7b-463d-bbb9-d4e0cecfa306" (UID: "133dc018-7d7b-463d-bbb9-d4e0cecfa306"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:33:15.461805 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.461783    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/133dc018-7d7b-463d-bbb9-d4e0cecfa306-kube-api-access-7t29j" (OuterVolumeSpecName: "kube-api-access-7t29j") pod "133dc018-7d7b-463d-bbb9-d4e0cecfa306" (UID: "133dc018-7d7b-463d-bbb9-d4e0cecfa306"). InnerVolumeSpecName "kube-api-access-7t29j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:33:15.561069 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.561015    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7t29j\" (UniqueName: \"kubernetes.io/projected/133dc018-7d7b-463d-bbb9-d4e0cecfa306-kube-api-access-7t29j\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:33:15.561069 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.561036    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/133dc018-7d7b-463d-bbb9-d4e0cecfa306-plugins-config-volume\") on node \"ip-10-0-131-77.ec2.internal\" DevicePath \"\""
Aug 05 18:33:15.586413 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.586387    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:33:15.785493 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.785472    2569 generic.go:356] "Generic (PLEG): container finished" podID="7ca7e691-e4ad-48a3-b3d3-e17d4858e002" containerID="5cd664d124081b4ec25b0089f08439cf68ef0a516d4a5f0046b0a938f9bc4c0c" exitCode=0
Aug 05 18:33:15.785619 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.785518    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"
Aug 05 18:33:15.785619 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.785549    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq" event={"ID":"7ca7e691-e4ad-48a3-b3d3-e17d4858e002","Type":"ContainerDied","Data":"5cd664d124081b4ec25b0089f08439cf68ef0a516d4a5f0046b0a938f9bc4c0c"}
Aug 05 18:33:15.785619 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.785604    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq" event={"ID":"7ca7e691-e4ad-48a3-b3d3-e17d4858e002","Type":"ContainerDied","Data":"84cd34ed46b84d41f1a652f6cf2f6c2d014cb806ee57e944fc85f9f73d448ca5"}
Aug 05 18:33:15.785758 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.785640    2569 scope.go:117] "RemoveContainer" containerID="5cd664d124081b4ec25b0089f08439cf68ef0a516d4a5f0046b0a938f9bc4c0c"
Aug 05 18:33:15.786985 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.786961    2569 generic.go:356] "Generic (PLEG): container finished" podID="133dc018-7d7b-463d-bbb9-d4e0cecfa306" containerID="2a463a8da7c41d026dc43dbbab7fc0fb713d40c8739aa6aece18d2682d4a6821" exitCode=0
Aug 05 18:33:15.787080 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.787001    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf" event={"ID":"133dc018-7d7b-463d-bbb9-d4e0cecfa306","Type":"ContainerDied","Data":"2a463a8da7c41d026dc43dbbab7fc0fb713d40c8739aa6aece18d2682d4a6821"}
Aug 05 18:33:15.787080 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.787013    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf"
Aug 05 18:33:15.787080 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.787049    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf" event={"ID":"133dc018-7d7b-463d-bbb9-d4e0cecfa306","Type":"ContainerDied","Data":"ab5411434da2766110fe18f86ba7623d28f5389fc1a772d525d648268b6aadeb"}
Aug 05 18:33:15.795429 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.795413    2569 scope.go:117] "RemoveContainer" containerID="5cd664d124081b4ec25b0089f08439cf68ef0a516d4a5f0046b0a938f9bc4c0c"
Aug 05 18:33:15.795720 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:33:15.795700    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cd664d124081b4ec25b0089f08439cf68ef0a516d4a5f0046b0a938f9bc4c0c\": container with ID starting with 5cd664d124081b4ec25b0089f08439cf68ef0a516d4a5f0046b0a938f9bc4c0c not found: ID does not exist" containerID="5cd664d124081b4ec25b0089f08439cf68ef0a516d4a5f0046b0a938f9bc4c0c"
Aug 05 18:33:15.795816 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.795729    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cd664d124081b4ec25b0089f08439cf68ef0a516d4a5f0046b0a938f9bc4c0c"} err="failed to get container status \"5cd664d124081b4ec25b0089f08439cf68ef0a516d4a5f0046b0a938f9bc4c0c\": rpc error: code = NotFound desc = could not find container \"5cd664d124081b4ec25b0089f08439cf68ef0a516d4a5f0046b0a938f9bc4c0c\": container with ID starting with 5cd664d124081b4ec25b0089f08439cf68ef0a516d4a5f0046b0a938f9bc4c0c not found: ID does not exist"
Aug 05 18:33:15.795816 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.795745    2569 scope.go:117] "RemoveContainer" containerID="2a463a8da7c41d026dc43dbbab7fc0fb713d40c8739aa6aece18d2682d4a6821"
Aug 05 18:33:15.803755 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.803739    2569 scope.go:117] "RemoveContainer" containerID="2a463a8da7c41d026dc43dbbab7fc0fb713d40c8739aa6aece18d2682d4a6821"
Aug 05 18:33:15.804001 ip-10-0-131-77 kubenswrapper[2569]: E0805 18:33:15.803983    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a463a8da7c41d026dc43dbbab7fc0fb713d40c8739aa6aece18d2682d4a6821\": container with ID starting with 2a463a8da7c41d026dc43dbbab7fc0fb713d40c8739aa6aece18d2682d4a6821 not found: ID does not exist" containerID="2a463a8da7c41d026dc43dbbab7fc0fb713d40c8739aa6aece18d2682d4a6821"
Aug 05 18:33:15.804042 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.804009    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a463a8da7c41d026dc43dbbab7fc0fb713d40c8739aa6aece18d2682d4a6821"} err="failed to get container status \"2a463a8da7c41d026dc43dbbab7fc0fb713d40c8739aa6aece18d2682d4a6821\": rpc error: code = NotFound desc = could not find container \"2a463a8da7c41d026dc43dbbab7fc0fb713d40c8739aa6aece18d2682d4a6821\": container with ID starting with 2a463a8da7c41d026dc43dbbab7fc0fb713d40c8739aa6aece18d2682d4a6821 not found: ID does not exist"
Aug 05 18:33:15.813062 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.813016    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"]
Aug 05 18:33:15.817004 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.816984    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6fb560-769d5c9cb-nlhlq"]
Aug 05 18:33:15.828700 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.828675    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf"]
Aug 05 18:33:15.834804 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:15.834786    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-6fb560-7bf88d497b-mrnnf"]
Aug 05 18:33:17.151370 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:17.151325    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="133dc018-7d7b-463d-bbb9-d4e0cecfa306" path="/var/lib/kubelet/pods/133dc018-7d7b-463d-bbb9-d4e0cecfa306/volumes"
Aug 05 18:33:17.151947 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:17.151927    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ca7e691-e4ad-48a3-b3d3-e17d4858e002" path="/var/lib/kubelet/pods/7ca7e691-e4ad-48a3-b3d3-e17d4858e002/volumes"
Aug 05 18:33:22.188800 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:22.188756    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:33:22.265921 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:22.265885    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:33:22.516108 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:22.516027    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:33:22.735066 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:22.735028    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:33:23.157691 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:23.157658    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5c844f45b7-mwq9n_b8d3e1c1-116a-43a7-b8f2-ba006085541d/authorino/0.log"
Aug 05 18:33:28.835651 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:28.835622    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67657dfbc-g4pbp_2c3bca01-3915-4c42-8478-39f4b3a87950/manager/0.log"
Aug 05 18:33:29.444161 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:29.444135    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-856dccdfcf-lbzpc_db10024a-4454-49c0-aa8a-c388c8290bf4/copy-manifests/0.log"
Aug 05 18:33:29.454769 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:29.454744    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-856dccdfcf-lbzpc_db10024a-4454-49c0-aa8a-c388c8290bf4/manager/0.log"
Aug 05 18:33:29.788159 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:29.788081    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5888db464-dll25_8c26998e-fd41-41dd-9530-0b7139f5372a/maas-api/0.log"
Aug 05 18:33:30.989092 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:30.989061    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5c844f45b7-mwq9n_b8d3e1c1-116a-43a7-b8f2-ba006085541d/authorino/0.log"
Aug 05 18:33:31.216233 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:31.216197    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-2p7dn_f328e280-a2cb-48e0-8c9f-10e2bc7af65b/manager/0.log"
Aug 05 18:33:31.447434 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:31.442887    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-t7hxb_48cf115d-5c47-4346-9d57-9a8117839e13/manager/0.log"
Aug 05 18:33:32.247506 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:32.247479    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57558c7678-8bjnz_fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170/kube-auth-proxy/0.log"
Aug 05 18:33:32.604473 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:32.604441    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-5phph_57db4d70-23ff-467d-a092-477df5cc8365/payload-pre-processing/0.log"
Aug 05 18:33:32.717255 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:32.717223    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-6tnnr_7b205b96-927c-4170-a375-5190000f495a/payload-processing/0.log"
Aug 05 18:33:33.514269 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:33.514235    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgthf9_6417b863-e45b-460b-9e49-1aae53077cd5/main/0.log"
Aug 05 18:33:33.741401 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:33.741367    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-rm2g6_e83ba2d0-0799-4dd8-9f0e-72a321adf79e/main/0.log"
Aug 05 18:33:38.734041 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:38.734002    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:33:39.859903 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:39.859869    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log" limit=1024
Aug 05 18:33:41.561143 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:41.561114    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2z2wd_bc73ab72-c095-4e0d-bbd9-b5296cadc187/global-pull-secret-syncer/0.log"
Aug 05 18:33:41.711465 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:41.711435    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4tkxv_a043e58e-b56d-4a56-b92b-7fb8f70694bc/konnectivity-agent/0.log"
Aug 05 18:33:41.850776 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:41.850731    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-77.ec2.internal_beae357a1d1b622108905e09a0f5f3aa/haproxy/0.log"
Aug 05 18:33:47.255071 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:47.255036    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5c844f45b7-mwq9n_b8d3e1c1-116a-43a7-b8f2-ba006085541d/authorino/0.log"
Aug 05 18:33:47.333338 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:47.333307    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-2p7dn_f328e280-a2cb-48e0-8c9f-10e2bc7af65b/manager/0.log"
Aug 05 18:33:47.454550 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:47.454455    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-t7hxb_48cf115d-5c47-4346-9d57-9a8117839e13/manager/0.log"
Aug 05 18:33:49.109954 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.109928    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ae4e325-445d-4cf5-acc1-40ccd5bff3ed/alertmanager/0.log"
Aug 05 18:33:49.136531 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.136509    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ae4e325-445d-4cf5-acc1-40ccd5bff3ed/config-reloader/0.log"
Aug 05 18:33:49.159983 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.159963    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ae4e325-445d-4cf5-acc1-40ccd5bff3ed/kube-rbac-proxy-web/0.log"
Aug 05 18:33:49.188757 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.188695    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ae4e325-445d-4cf5-acc1-40ccd5bff3ed/kube-rbac-proxy/0.log"
Aug 05 18:33:49.213837 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.213818    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ae4e325-445d-4cf5-acc1-40ccd5bff3ed/kube-rbac-proxy-metric/0.log"
Aug 05 18:33:49.239560 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.239539    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ae4e325-445d-4cf5-acc1-40ccd5bff3ed/prom-label-proxy/0.log"
Aug 05 18:33:49.265134 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.265115    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ae4e325-445d-4cf5-acc1-40ccd5bff3ed/init-config-reloader/0.log"
Aug 05 18:33:49.346362 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.346336    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-855bm_8721c842-6692-47bd-a37a-3c1d48edc59a/kube-state-metrics/0.log"
Aug 05 18:33:49.367825 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.367801    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-855bm_8721c842-6692-47bd-a37a-3c1d48edc59a/kube-rbac-proxy-main/0.log"
Aug 05 18:33:49.393783 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.393765    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-855bm_8721c842-6692-47bd-a37a-3c1d48edc59a/kube-rbac-proxy-self/0.log"
Aug 05 18:33:49.426750 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.426729    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-798899d854-2rwbz_81a88269-e4c5-4800-9f53-45cee2b243bc/metrics-server/0.log"
Aug 05 18:33:49.458618 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.458558    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-6dffbbd67-pvjkg_624e4dca-7205-45fa-b44d-5d09f307468f/monitoring-plugin/0.log"
Aug 05 18:33:49.671069 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.671045    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w69vr_2c558452-bf1d-44ba-a169-a78f38dc4bb3/node-exporter/0.log"
Aug 05 18:33:49.696188 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.696163    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w69vr_2c558452-bf1d-44ba-a169-a78f38dc4bb3/kube-rbac-proxy/0.log"
Aug 05 18:33:49.720650 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.720595    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w69vr_2c558452-bf1d-44ba-a169-a78f38dc4bb3/init-textfile/0.log"
Aug 05 18:33:49.747749 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.747732    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-vmbsz_9ee151b3-b9ea-499f-afd4-816e849c1c5f/kube-rbac-proxy-main/0.log"
Aug 05 18:33:49.780316 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.780295    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-vmbsz_9ee151b3-b9ea-499f-afd4-816e849c1c5f/kube-rbac-proxy-self/0.log"
Aug 05 18:33:49.804162 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.804141    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-vmbsz_9ee151b3-b9ea-499f-afd4-816e849c1c5f/openshift-state-metrics/0.log"
Aug 05 18:33:49.867266 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.867222    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d0389b49-02c0-4ec9-82fd-e47ceb75ceca/prometheus/0.log"
Aug 05 18:33:49.891137 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.891106    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d0389b49-02c0-4ec9-82fd-e47ceb75ceca/config-reloader/0.log"
Aug 05 18:33:49.916706 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.916678    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d0389b49-02c0-4ec9-82fd-e47ceb75ceca/thanos-sidecar/0.log"
Aug 05 18:33:49.942417 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.942390    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d0389b49-02c0-4ec9-82fd-e47ceb75ceca/kube-rbac-proxy-web/0.log"
Aug 05 18:33:49.968662 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.968636    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d0389b49-02c0-4ec9-82fd-e47ceb75ceca/kube-rbac-proxy/0.log"
Aug 05 18:33:49.993800 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:49.993735    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d0389b49-02c0-4ec9-82fd-e47ceb75ceca/kube-rbac-proxy-thanos/0.log"
Aug 05 18:33:50.033054 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:50.033027    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d0389b49-02c0-4ec9-82fd-e47ceb75ceca/init-config-reloader/0.log"
Aug 05 18:33:50.084923 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:50.084891    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-fdxhn_fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b/prometheus-operator/0.log"
Aug 05 18:33:50.109147 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:50.109118    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-fdxhn_fa95a02b-aa5c-4a7b-8e37-1a18c74ad92b/kube-rbac-proxy/0.log"
Aug 05 18:33:50.136954 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:50.136922    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-66685ff6d4-q5dht_22a6cba0-5553-42d8-b6c6-83a5c261eb34/prometheus-operator-admission-webhook/0.log"
Aug 05 18:33:50.254029 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:50.253950    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d896f6548-kx6m8_5221e25b-9034-4392-a945-2fce3cb7f40e/thanos-query/0.log"
Aug 05 18:33:50.278324 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:50.278299    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d896f6548-kx6m8_5221e25b-9034-4392-a945-2fce3cb7f40e/kube-rbac-proxy-web/0.log"
Aug 05 18:33:50.302927 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:50.302904    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d896f6548-kx6m8_5221e25b-9034-4392-a945-2fce3cb7f40e/kube-rbac-proxy/0.log"
Aug 05 18:33:50.325934 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:50.325907    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d896f6548-kx6m8_5221e25b-9034-4392-a945-2fce3cb7f40e/prom-label-proxy/0.log"
Aug 05 18:33:50.348420 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:50.348389    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d896f6548-kx6m8_5221e25b-9034-4392-a945-2fce3cb7f40e/kube-rbac-proxy-rules/0.log"
Aug 05 18:33:50.374921 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:50.374906    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d896f6548-kx6m8_5221e25b-9034-4392-a945-2fce3cb7f40e/kube-rbac-proxy-metrics/0.log"
Aug 05 18:33:51.474650 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.474613    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"]
Aug 05 18:33:51.475428 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475031    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="133dc018-7d7b-463d-bbb9-d4e0cecfa306" containerName="payload-pre-processing"
Aug 05 18:33:51.475428 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475043    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="133dc018-7d7b-463d-bbb9-d4e0cecfa306" containerName="payload-pre-processing"
Aug 05 18:33:51.475428 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475054    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2348a2eb-64b8-4e73-8f2a-7e0aacbf8399" containerName="main"
Aug 05 18:33:51.475428 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475059    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2348a2eb-64b8-4e73-8f2a-7e0aacbf8399" containerName="main"
Aug 05 18:33:51.475428 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475072    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2473b44-6051-4d8e-b17a-349d85441ae6" containerName="payload-pre-processing"
Aug 05 18:33:51.475428 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475081    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2473b44-6051-4d8e-b17a-349d85441ae6" containerName="payload-pre-processing"
Aug 05 18:33:51.475428 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475095    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77e76780-cf13-4569-ba5a-43bcb5409418" containerName="maas-api"
Aug 05 18:33:51.475428 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475104    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="77e76780-cf13-4569-ba5a-43bcb5409418" containerName="maas-api"
Aug 05 18:33:51.475428 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475128    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10015b7a-34e1-4c8a-be41-f9c63e942cfe" containerName="maas-api"
Aug 05 18:33:51.475428 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475135    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="10015b7a-34e1-4c8a-be41-f9c63e942cfe" containerName="maas-api"
Aug 05 18:33:51.475428 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475280    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ebad985-3d98-447d-ad95-a868108e610c" containerName="maas-api"
Aug 05 18:33:51.475428 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475290    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ebad985-3d98-447d-ad95-a868108e610c" containerName="maas-api"
Aug 05 18:33:51.475428 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475312    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e746138a-5050-4ec5-98db-304839bffc5c" containerName="payload-processing"
Aug 05 18:33:51.475428 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475317    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e746138a-5050-4ec5-98db-304839bffc5c" containerName="payload-processing"
Aug 05 18:33:51.475428 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475326    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db" containerName="payload-processing"
Aug 05 18:33:51.475428 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475331    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db" containerName="payload-processing"
Aug 05 18:33:51.475428 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475338    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ca7e691-e4ad-48a3-b3d3-e17d4858e002" containerName="maas-api"
Aug 05 18:33:51.475428 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475343    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ca7e691-e4ad-48a3-b3d3-e17d4858e002" containerName="maas-api"
Aug 05 18:33:51.476099 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475446    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2348a2eb-64b8-4e73-8f2a-7e0aacbf8399" containerName="main"
Aug 05 18:33:51.476099 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475455    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc805f0e-dda3-44b6-bc7b-9a7b6e73a7db" containerName="payload-processing"
Aug 05 18:33:51.476099 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475461    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="10015b7a-34e1-4c8a-be41-f9c63e942cfe" containerName="maas-api"
Aug 05 18:33:51.476099 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475468    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="77e76780-cf13-4569-ba5a-43bcb5409418" containerName="maas-api"
Aug 05 18:33:51.476099 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475476    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ca7e691-e4ad-48a3-b3d3-e17d4858e002" containerName="maas-api"
Aug 05 18:33:51.476099 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475482    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="133dc018-7d7b-463d-bbb9-d4e0cecfa306" containerName="payload-pre-processing"
Aug 05 18:33:51.476099 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475488    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2473b44-6051-4d8e-b17a-349d85441ae6" containerName="payload-pre-processing"
Aug 05 18:33:51.476099 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475494    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e746138a-5050-4ec5-98db-304839bffc5c" containerName="payload-processing"
Aug 05 18:33:51.476099 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.475500    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ebad985-3d98-447d-ad95-a868108e610c" containerName="maas-api"
Aug 05 18:33:51.478866 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.478851    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"
Aug 05 18:33:51.482685 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.482664    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rl9xm\"/\"kube-root-ca.crt\""
Aug 05 18:33:51.483266 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.483250    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rl9xm\"/\"openshift-service-ca.crt\""
Aug 05 18:33:51.483360 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.483255    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rl9xm\"/\"default-dockercfg-62k6z\""
Aug 05 18:33:51.488921 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.488902    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"]
Aug 05 18:33:51.587004 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.586970    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930-proc\") pod \"perf-node-gather-daemonset-k67xk\" (UID: \"cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930\") " pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"
Aug 05 18:33:51.587173 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.587021    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930-sys\") pod \"perf-node-gather-daemonset-k67xk\" (UID: \"cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930\") " pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"
Aug 05 18:33:51.587173 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.587070    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930-podres\") pod \"perf-node-gather-daemonset-k67xk\" (UID: \"cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930\") " pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"
Aug 05 18:33:51.587173 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.587124    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930-lib-modules\") pod \"perf-node-gather-daemonset-k67xk\" (UID: \"cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930\") " pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"
Aug 05 18:33:51.587173 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.587140    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-884td\" (UniqueName: \"kubernetes.io/projected/cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930-kube-api-access-884td\") pod \"perf-node-gather-daemonset-k67xk\" (UID: \"cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930\") " pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"
Aug 05 18:33:51.688023 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.687988    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930-proc\") pod \"perf-node-gather-daemonset-k67xk\" (UID: \"cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930\") " pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"
Aug 05 18:33:51.688224 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.688054    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930-sys\") pod \"perf-node-gather-daemonset-k67xk\" (UID: \"cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930\") " pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"
Aug 05 18:33:51.688224 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.688090    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930-podres\") pod \"perf-node-gather-daemonset-k67xk\" (UID: \"cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930\") " pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"
Aug 05 18:33:51.688224 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.688117    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930-proc\") pod \"perf-node-gather-daemonset-k67xk\" (UID: \"cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930\") " pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"
Aug 05 18:33:51.688224 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.688124    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930-sys\") pod \"perf-node-gather-daemonset-k67xk\" (UID: \"cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930\") " pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"
Aug 05 18:33:51.688224 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.688136    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930-lib-modules\") pod \"perf-node-gather-daemonset-k67xk\" (UID: \"cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930\") " pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"
Aug 05 18:33:51.688224 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.688160    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-884td\" (UniqueName: \"kubernetes.io/projected/cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930-kube-api-access-884td\") pod \"perf-node-gather-daemonset-k67xk\" (UID: \"cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930\") " pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"
Aug 05 18:33:51.688455 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.688253    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930-podres\") pod \"perf-node-gather-daemonset-k67xk\" (UID: \"cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930\") " pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"
Aug 05 18:33:51.688455 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.688307    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930-lib-modules\") pod \"perf-node-gather-daemonset-k67xk\" (UID: \"cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930\") " pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"
Aug 05 18:33:51.698418 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.698398    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-884td\" (UniqueName: \"kubernetes.io/projected/cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930-kube-api-access-884td\") pod \"perf-node-gather-daemonset-k67xk\" (UID: \"cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930\") " pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"
Aug 05 18:33:51.788967 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.788878    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"
Aug 05 18:33:51.928604 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.928576    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"]
Aug 05 18:33:51.933522 ip-10-0-131-77 kubenswrapper[2569]: W0805 18:33:51.933491    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcb56f5c9_d4fc_4877_8d0d_1e1bedcdc930.slice/crio-04c0afda663a78d4359b781d7ec7557c6501fc5d4831b82b62f253ed8f61f938 WatchSource:0}: Error finding container 04c0afda663a78d4359b781d7ec7557c6501fc5d4831b82b62f253ed8f61f938: Status 404 returned error can't find the container with id 04c0afda663a78d4359b781d7ec7557c6501fc5d4831b82b62f253ed8f61f938
Aug 05 18:33:51.965353 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:51.965319    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk" event={"ID":"cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930","Type":"ContainerStarted","Data":"04c0afda663a78d4359b781d7ec7557c6501fc5d4831b82b62f253ed8f61f938"}
Aug 05 18:33:52.874611 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:52.874583    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7646bfbd48-mhvks_737049a5-b68e-4c3e-ab9b-87aa444304c8/console/0.log"
Aug 05 18:33:52.972046 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:52.972010    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk" event={"ID":"cb56f5c9-d4fc-4877-8d0d-1e1bedcdc930","Type":"ContainerStarted","Data":"946dd1e91761f5fa364ff16d1b8895fdd640a840c00d38428add1d2d83d2e762"}
Aug 05 18:33:52.972283 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:52.972158    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"
Aug 05 18:33:52.992262 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:52.992208    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk" podStartSLOduration=1.992174231 podStartE2EDuration="1.992174231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 18:33:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:33:52.991105438 +0000 UTC m=+3640.343780034" watchObservedRunningTime="2026-08-05 18:33:52.992174231 +0000 UTC m=+3640.344848825"
Aug 05 18:33:54.768860 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:54.768822    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ntkvz_b92828f0-7eb1-4149-acad-b21d6845bf04/dns/0.log"
Aug 05 18:33:54.792272 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:54.792256    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ntkvz_b92828f0-7eb1-4149-acad-b21d6845bf04/kube-rbac-proxy/0.log"
Aug 05 18:33:54.896741 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:54.896709    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-v8pbg_edf39462-7bb0-4c83-8979-c71a9df1caba/dns-node-resolver/0.log"
Aug 05 18:33:55.587356 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:55.587322    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4jm79_13b7a2b4-3185-4eae-98d8-855f2325e268/node-ca/0.log"
Aug 05 18:33:56.946233 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:56.946195    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57558c7678-8bjnz_fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170/kube-auth-proxy/0.log"
Aug 05 18:33:57.056321 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:57.056286    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-5phph_57db4d70-23ff-467d-a092-477df5cc8365/payload-pre-processing/0.log"
Aug 05 18:33:57.103486 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:57.103454    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-6tnnr_7b205b96-927c-4170-a375-5190000f495a/payload-processing/0.log"
Aug 05 18:33:58.016556 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:58.016515    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-czglh_d6e3eb00-c647-435d-aced-22a116ccc2b7/serve-healthcheck-canary/0.log"
Aug 05 18:33:58.725492 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:58.725453    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s79s8_cb27b329-ae29-450d-9029-f24d7840c1dc/kube-rbac-proxy/0.log"
Aug 05 18:33:58.747128 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:58.747101    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s79s8_cb27b329-ae29-450d-9029-f24d7840c1dc/exporter/0.log"
Aug 05 18:33:58.774709 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:58.774687    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s79s8_cb27b329-ae29-450d-9029-f24d7840c1dc/extractor/0.log"
Aug 05 18:33:58.988014 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:33:58.987943    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rl9xm/perf-node-gather-daemonset-k67xk"
Aug 05 18:34:02.156629 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:34:02.156600    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67657dfbc-g4pbp_2c3bca01-3915-4c42-8478-39f4b3a87950/manager/0.log"
Aug 05 18:34:02.390896 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:34:02.390868    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57558c7678-8bjnz_fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170/kube-auth-proxy/0.log"
Aug 05 18:34:02.496571 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:34:02.496498    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-5phph_57db4d70-23ff-467d-a092-477df5cc8365/payload-pre-processing/0.log"
Aug 05 18:34:02.524074 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:34:02.524052    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-6tnnr_7b205b96-927c-4170-a375-5190000f495a/payload-processing/0.log"
Aug 05 18:34:02.708036 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:34:02.708017    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-856dccdfcf-lbzpc_db10024a-4454-49c0-aa8a-c388c8290bf4/manager/0.log"
Aug 05 18:34:02.751545 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:34:02.751491    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-856dccdfcf-lbzpc_db10024a-4454-49c0-aa8a-c388c8290bf4/copy-manifests/0.log"
Aug 05 18:34:04.072098 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:34:04.072070    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-dd9d7cdcd-mwfrd_40522c5b-a3f8-4343-9755-0795762c9808/manager/0.log"
Aug 05 18:34:04.212403 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:34:04.212376    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57558c7678-8bjnz_fa67e0f0-fbbf-4d26-8d7e-1d8b98b9d170/kube-auth-proxy/0.log"
Aug 05 18:34:04.329134 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:34:04.329071    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-5phph_57db4d70-23ff-467d-a092-477df5cc8365/payload-pre-processing/0.log"
Aug 05 18:34:04.355062 ip-10-0-131-77 kubenswrapper[2569]: I0805 18:34:04.355043    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-6tnnr_7b205b96-927c-4170-a375-5190000f495a/payload-processing/0.log"